question Re: What is the difference between Google Dataflow and Hadoop Data Flow in Archives of Support Questions (Read Only)

What is the difference between Google Dataflow and Hadoop Data Flow

mherring — Tue, 21 Apr 2026 12:33:05 GMT

Can someone please help me understand the difference between Google Cloud DataFlow and Hortonworks DataFlow. Are there technical differences I should be aware of?

Re: What is the difference between Google Dataflow and Hadoop Data Flow

hliu — Sat, 23 Jan 2016 07:47:25 GMT

Google Cloud Dataflow is a service which replaces MapReduce processing, and is designed strictly for the Google Compute Cloud. Whereas Hortonworks Dataflow is a product aiming to solve data flow problems, even outside of data center.

So the answer is no, they are essentially using similar names to describe very different things. One is sitting in the cloud waiting for data to be delivered to it; and the other one delivers data to all kinds of processing systems: Google Dataflow, Storm, Spark, etc.

Re: What is the difference between Google Dataflow and Hadoop Data Flow

aervits — Sat, 23 Jan 2016 10:04:33 GMT

Google Dataflow is a language framework for multiple engines like Spark, Flink and mapreduce. Hadoop Data Flow is a data in motion processing tool with a visual editor.

Re: What is the difference between Google Dataflow and Hadoop Data Flow

marko_kole — Thu, 04 Feb 2016 21:36:21 GMT

More an info than an answer: proposal to join the apache incubator:

http://googlecloudplatform.blogspot.ch/2016/01/Dataflow-and-open-source-proposal-to-join-the-Apache-Incubator.html

acceptance:

https://wiki.apache.org/incubator/BeamProposal