Amazon EMR is an amazing value proposition for virtually any research need, and it's very cool to see wrapper frameworks targeting it directly. Still, for anyone managing their own compute clusters and wanting to do MR in python, I'd suggest checking out Disco.
Disco (http://discoproject.org) is a really elegant MR framework implemented in erlang and python, with additional support for jobs in C and Java. I've used it for a little over a year and am convinced it is the superior MR platform (Hadoop's terasort victories notwithstanding). New features are being integrated very quickly, the core platform is rock solid, management is simple and it's extremely flexible.
Comments
Amazon EMR is an amazing value proposition for virtually any research need, and it's very cool to see wrapper frameworks targeting it directly. Still, for anyone managing their own compute clusters and wanting to do MR in python, I'd suggest checking out Disco.
Disco (http://discoproject.org) is a really elegant MR framework implemented in erlang and python, with additional support for jobs in C and Java. I've used it for a little over a year and am convinced it is the superior MR platform (Hadoop's terasort victories notwithstanding). New features are being integrated very quickly, the core platform is rock solid, management is simple and it's extremely flexible.