https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132154#M31438 <P><A href="http://kafka.apache.org/documentation.html#configuration" target="_blank">http://kafka.apache.org/documentation.html#configuration</A></P> Fri, 10 Jun 2016 22:32:47 GMT TimothySpann 2016-06-10T22:32:47Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132149#M31433 <P>I want to install kafka on my cluster and use local storage for kafka instead of hdfs. Is this setup possible through ambari?</P> Fri, 10 Jun 2016 11:25:46 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132149#M31433 sunile_manjee 2016-06-10T11:25:46Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132150#M31434 <P><A rel="user" href="https://community.cloudera.com/users/1486/smanjee.html" nodeid="1486">@Sunile Manjee</A> </P><P>I did not follow. I guess Kafka uses local storage anyway for log storage not hdfs.</P><P>Can you share more details?</P> Fri, 10 Jun 2016 11:38:27 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132150#M31434 rpathak 2016-06-10T11:38:27Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132151#M31435 <P style="margin-left: 40px;"> <A rel="user" href="https://community.cloudera.com/users/872/rahulpathak109.html" nodeid="872">@Rahul Pathak</A> I was not aware that was the only option. I was under the impression kafka can use local or hdfs storage for log storage. So clearly my assumption was wrong.</P> Fri, 10 Jun 2016 11:50:01 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132151#M31435 sunile_manjee 2016-06-10T11:50:01Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132152#M31436 <P style="margin-left: 40px;"> <A rel="user" href="https://community.cloudera.com/users/872/rahulpathak109.html" nodeid="872">@Rahul Pathak</A> My ? may not be applicable due to my mis-understanding kafka storing logs on HDFS as well as local. Do i understand you correctly that kafka only stores logs to local disk?</P> Fri, 10 Jun 2016 12:12:32 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132152#M31436 sunile_manjee 2016-06-10T12:12:32Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132153#M31437 <P>Yes kafka store its log at local disk.</P><P>To push kafka logs to hdfs you need something like <A href="http://gobblin.readthedocs.io/en/latest/case-studies/Kafka-HDFS-Ingestion/" target="_blank">http://gobblin.readthedocs.io/en/latest/case-studies/Kafka-HDFS-Ingestion/</A></P> Fri, 10 Jun 2016 14:29:33 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132153#M31437 rpathak 2016-06-10T14:29:33Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132154#M31438 <P><A href="http://kafka.apache.org/documentation.html#configuration" target="_blank">http://kafka.apache.org/documentation.html#configuration</A></P> Fri, 10 Jun 2016 22:32:47 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-install-kafka-through-ambari-without-HDFS/m-p/132154#M31438 TimothySpann 2016-06-10T22:32:47Z