What is the difference between Google Dataflow and Hadoop Data Flow

[#1]

Can someone please help me understand the difference between Google Cloud DataFlow and Hortonworks DataFlow. Are there technical differences I should be aware of?

[#2 · Accepted solution]

Google Cloud Dataflow is a service which replaces MapReduce processing, and is designed strictly for the Google Compute Cloud. Whereas Hortonworks Dataflow is a product aiming to solve data flow problems, even outside of data center.

So the answer is no, they are essentially using similar names to describe very different things. One is sitting in the cloud waiting for data to be delivered to it; and the other one delivers data to all kinds of processing systems: Google Dataflow, Storm, Spark, etc.

[#3]

Google Dataflow is a language framework for multiple engines like Spark, Flink and mapreduce. Hadoop Data Flow is a data in motion processing tool with a visual editor.

[#4]

More an info than an answer: proposal to join the apache incubator:

http://googlecloudplatform.blogspot.ch/2016/01/Dataflow-and-open-source-proposal-to-join-the-Apache-...

acceptance:

https://wiki.apache.org/incubator/BeamProposal