Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What is the old legacy kafka-clients-0.8 for in HDP 3.0?

What is the old legacy kafka-clients-0.8 for in HDP 3.0?

New Contributor

I'm trying to connect with HDP 3.0 built-in kafka 1.0.1 using consumer/producer API.

When I run the kafka consumers on several cluster nodes, I found an issue with incompatililty between kafka client/server. I'm using the correct kafka 1.0.1 client jars, but I found another legacy "kafka-clients-0.8.2.1.jar" added into classpath, which is causing incompatibility issue:

2CIUSERARG              
-Djava.class.path=/opt/IBM/InformationServer/Server/DSEngine/../DSComponents/bin/ccapi.jar:.........:/usr/hdp/3.0.1.0-187/hadoop-mapreduce/.//kafka-clients-0.8.2.1.jar:......
2CIENVVAR      
CLASSPATH=/usr/hdp/3.0.1.0-187/hadoop/conf:/usr/hdp/3.0.1.0-187/hadoop/lib/nimbus-jose-jwt-4.41.1.jar:.......:/usr/hdp/3.0.1.0-187/hadoop-mapreduce/.//kafka-clients-0.8.2.1.jar:......:/usr/hdp/3.0.1.0-187/tez/lib/tez.tar.gz:./AppMaster.jar

I found that this jar is being added to java process using script:

/usr/hdp/3.0.0.0-1634/hadoop/libexec/shellprofile.d/hadoop-kafka.sh:

    hadoop_add_classpath "${HADOOP_TOOLS_HOME}/${HADOOP_TOOLS_LIB_JARS_DIR}/kafka-clients-0.8.2.1.jar"

The question is: what is this old legacy kafka-clients-0.8.2.1.jar used for? It has been added in HDP v3., for 2.x version it wasn't there

Don't have an account?
Coming from Hortonworks? Activate your account here