Dealing With Excel Data in PySpark

Have you ever asked yourself, "how do I read in 10,000 Excel files and process them using Spark?" I hope sounds like a terrible task...but in case you have, it just so happens I might have an approach your interested in.

General Approach

PySpark does not …

Perhaps You've Noticed: I Have a New Blog

Yep, that's right...I kicked WordPress to the curb and decided to do the whole staticly-generated-pages-thing. When I originally started this blog, WordPress just made the most sense. It was easy to install, easy to maintain, had plenty of skins and plugins available, and just worked.

So, why the departure …