Reply
Explorer
Posts: 25
Registered: ‎07-11-2016

Kafka on Edge Node in cloudera cluster

Hi,

 

We will be creating a Hadoop cluster with Edge node using cloudera manager 5.8. We will setup spark gateway, Impala Daemon, HDFS Gateway, Hive gateway and Hue Server on Edge node to communicate with master nodes. We have  a query related to Kafka, We have setup Kafka Brokers on 3 servers but not on Edge node as it will be wrong in principal.

 

How can we setup kafka client libs on edge node as we will be running the consumer from Edge node as well as will run the adhoc kafka commands like[/usr/bin/kafka-topics --list --zookeeper IP:2181] . Do we need to simply copy the kafka files[e.g. kafka-topics, kafka-console-consumer] under /usr/bin/ to the edge node at same location?

 

Can you please help us on the same.

 

Thanks,

Amit

Explorer
Posts: 25
Registered: ‎07-11-2016

Re: Kafka on Edge Node in cloudera cluster

Hi, can anybody please help me on the same.

Thanks,
Amit
Highlighted
Cloudera Employee
Posts: 4
Registered: ‎07-27-2016

Re: Kafka on Edge Node in cloudera cluster

This seems to be a kakfa specific issue. I suggest you post your question to Data Ingestions & Integration (Apache Kafka, Apache Sqoop, Apache Flume...), which is a better forum to cover Kafka specific questions.