https://community.cloudera.com/t5/Archives-of-Support-Questions/where-to-install-kafka/m-p/167617#M41615 <P>This is a great guide to what gets installed where on HDP: <A href="https://community.hortonworks.com/articles/16763/cheat-sheet-and-tips-for-a-custom-install-of-horto.html" target="_blank">https://community.hortonworks.com/articles/16763/cheat-sheet-and-tips-for-a-custom-install-of-horto.html</A></P><P>You will notice that Kafka should be installed within the cluster and is best dedicated to its own nodes.</P><P>As a side note, Hortwonworks Data Flow (HDF) is a separate distribution/product provided by Hortonworks. It packages Kafka along with NiFi, Storm and Ambari and excels at acquiring, inspecting, routing, transforming, analyizing data in motion from a diverse number of sources (ranging from sensors to databases), which is typically outputted in Hadoop. Exciting technology and a lot to talk ... check it out: <A href="http://hortonworks.com/products/data-center/hdf/" target="_blank">http://hortonworks.com/products/data-center/hdf/</A> </P> Fri, 23 Sep 2016 19:36:42 GMT gkeys 2016-09-23T19:36:42Z https://community.cloudera.com/t5/Archives-of-Support-Questions/where-to-install-kafka/m-p/167616#M41614 <P>Hi,</P><P>Just wondering how the cluster topology should look like for Kafka alongside Hadoop? I presume Kafka brokers shouldn't be co-located alongside data nodes. Instead should probably be installed on nodes outside Hadoop cluster (probably gateway / edge nodes) as Kafka serves as the landing area and the data be eventually pushed to one of the Hadoop storage engines.</P><P>Am I correct thinking this way? Please validate my understanding.</P> Fri, 23 Sep 2016 17:52:21 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/where-to-install-kafka/m-p/167616#M41614 bigdata_superno 2016-09-23T17:52:21Z https://community.cloudera.com/t5/Archives-of-Support-Questions/where-to-install-kafka/m-p/167617#M41615 <P>This is a great guide to what gets installed where on HDP: <A href="https://community.hortonworks.com/articles/16763/cheat-sheet-and-tips-for-a-custom-install-of-horto.html" target="_blank">https://community.hortonworks.com/articles/16763/cheat-sheet-and-tips-for-a-custom-install-of-horto.html</A></P><P>You will notice that Kafka should be installed within the cluster and is best dedicated to its own nodes.</P><P>As a side note, Hortwonworks Data Flow (HDF) is a separate distribution/product provided by Hortonworks. It packages Kafka along with NiFi, Storm and Ambari and excels at acquiring, inspecting, routing, transforming, analyizing data in motion from a diverse number of sources (ranging from sensors to databases), which is typically outputted in Hadoop. Exciting technology and a lot to talk ... check it out: <A href="http://hortonworks.com/products/data-center/hdf/" target="_blank">http://hortonworks.com/products/data-center/hdf/</A> </P> Fri, 23 Sep 2016 19:36:42 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/where-to-install-kafka/m-p/167617#M41615 gkeys 2016-09-23T19:36:42Z