07-28-2016 02:05 AM
We will be creating a Hadoop cluster with Edge node using cloudera manager 5.8. We will setup spark gateway, Impala Daemon, HDFS Gateway, Hive gateway and Hue Server on Edge node to communicate with master nodes. We have a query related to Kafka, We have setup Kafka Brokers on 3 servers but not on Edge node as it will be wrong in principal.
How can we setup kafka client libs on edge node as we will be running the consumer from Edge node as well as will run the adhoc kafka commands like[/usr/bin/kafka-topics --list --zookeeper IP:2181] . Do we need to simply copy the kafka files[e.g. kafka-topics, kafka-console-consumer] under /usr/bin/ to the edge node at same location?
Can you please help us on the same.
08-01-2016 12:30 AM
This seems to be a kakfa specific issue. I suggest you post your question to Data Ingestions & Integration (Apache Kafka, Apache Sqoop, Apache Flume...), which is a better forum to cover Kafka specific questions.