Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

kafka connect

How to install kafka connect in a distributed mnode in HDP. I am using HDP 2.6



@ARUN Are you able to setup kafka-Connect with HDP 2.6. If so, can you please brief the steps.

Super Collaborator

If you are running Kafka 0.10 or newer, exists somewhere under /usr/hdp/current/kafka already.

You can run that process on multiple machines to create a Kafka Connect cluster.

New Contributor

What command did you use to run this and what is the location of location in hortonworks?




Kafka Connect Setup:

  1. Download the Confluent-Kafka tar for Confluent:
2. Untar the package and copy the '/share' folder under '/usr/hdp/hdp_version_/kafka/' folder
3. update the CLASSPPATH with jars files location, in my case its '/usr/hdp/'
4. Make appropriate changes to 'connect-distributed' & 'connect-standalone' property files under /etc/kafka/hdp_version/0/
5. I added '' under '/etc/kafka/hdp_version/0/' which includes topic names,topics dirs,flush size etc.
6. Run a test job with these changes and worked for me.

**Attaching a template of quickstart hdfs properties file.quickstart-hdfs.txt

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.