DataFlow is not Kinesis. It's more like Kinesis plus Esper plus BigQuery and you still wouldn't have one set of queries to run against streaming and batch data like you do with DataFlow.
I've been using Simple Workflow (in particular, the Flow framework: http://aws.amazon.com/swf/details/flow/) a lot recently to manage complex asynchronous distributed workflows, and it's a revelation. Some people laugh about the "Simple" part in the name, but it's kind of true - once you get over the initial learning curve. It's a bit like Git that way (First a lot of banging your head, then a lot of bang for buck)
Comments
What do you think is the analogous product to Cloud Dataflow in the AWS ecosystem? SWF? http://aws.amazon.com/swf/
I believe it's Kinesis: http://aws.amazon.com/kinesis/
+1. It's Kinesis.
DataFlow is not Kinesis. It's more like Kinesis plus Esper plus BigQuery and you still wouldn't have one set of queries to run against streaming and batch data like you do with DataFlow.
I presume AWS Data Pipeline. But they have their differences, so perhaps there is no true analog.
http://aws.amazon.com/datapipeline/
Simple Workflow is new to me, so thanks for putting it on my radar!
I've been using Simple Workflow (in particular, the Flow framework: http://aws.amazon.com/swf/details/flow/) a lot recently to manage complex asynchronous distributed workflows, and it's a revelation. Some people laugh about the "Simple" part in the name, but it's kind of true - once you get over the initial learning curve. It's a bit like Git that way (First a lot of banging your head, then a lot of bang for buck)