Because my crap has to live somewhere
This article is about a project that I recently finished and delivered. The point of this article isn't to talk about the technical nitty-gritty of how it works, or why it works, or how many cans of Monster it took me to complete it (hint: I drank a lot of …
Holy moly, it's been a while since I've written! Well, it's time to fix that. I've done a lot of out-of-band development and side projects since I've last written, but nothing that I've published until now.
For one of my side projects I'm using a Raspberry Pi 3B …
Have you ever asked yourself, "how do I read in 10,000 Excel files and process them using Spark?" I hope not...it sounds like a terrible task...but in case you have, it just so happens I might have an approach your interested in.
PySpark does not …
In my most recent role, we're using Python and Spark to perform a complex ETL process and to produce data that will ultimately be used to produce some model. During this process, we were using PySpark's pyspark.sql.types.DateType to store date information. At some point, we decided that …
Like most apps on the internet, the stuff that I write at Showroom Logic has scheduled tasks that must happen in a predictable fashion. In our case, we have some reports to run and deliver. Our app, like lots of internet apps, is distributed and runs in a Docker container …