Yep. This is exactly why I gravitated toward python and scikit-learn. So much of data science is just getting the data into the right format. Grab it from this file, and this database, and this web service, then get it formatted into this table structure, and then clean it with this filter, and then plug these holes this way and those other holes that way, and now you're finally ready for a random forest baseline.
Python is a really good language for merging and parsing data from lots of different sources. For many problems, a general purpose language with very good data science library may actually be the better choice than a dedicated data science language/environment.
Comments
"for gluing together my data pipeline"
Yep. This is exactly why I gravitated toward python and scikit-learn. So much of data science is just getting the data into the right format. Grab it from this file, and this database, and this web service, then get it formatted into this table structure, and then clean it with this filter, and then plug these holes this way and those other holes that way, and now you're finally ready for a random forest baseline.
Python is a really good language for merging and parsing data from lots of different sources. For many problems, a general purpose language with very good data science library may actually be the better choice than a dedicated data science language/environment.