Holy moly, it's been a while since I've written! Well, it's time to fix that. I've done a lot of out-of-band development and side projects since I've last written, but nothing that I've published until now.
For one of my side projects I'm using a Raspberry Pi 3B …
Have you ever asked yourself, "how do I read in 10,000 Excel files and process them using Spark?" I hope not...it sounds like a terrible task...but in case you have, it just so happens I might have an approach your interested in.
PySpark does not …
In my most recent role, we're using Python and Spark to perform a complex ETL process and to produce data that will ultimately be used to produce some model. During this process, we were using PySpark's pyspark.sql.types.DateType to store date information. At some point, we decided that …
Bottom line up front:
At my place of work, I'm in charge of the Data Pipeline team. Part of this team's job is to make data retrieval and processing reliable and flexible. After doing some thinking, we realized that solutions such as …
Yep, that's right...I kicked WordPress to the curb and decided to do the whole staticly-generated-pages-thing. When I originally started this blog, WordPress just made the most sense. It was easy to install, easy to maintain, had plenty of skins and plugins available, and just worked.
So, why the departure …