New Contributor
Posts: 5
Registered: ‎08-21-2018

Re: Console producer/consumer not working in kafka 0.10.2 with kerberos

[ Edited ]



Follow below steps.

1)change Inter Broker Protocol property to SASL_PLAINTEXT in Cloudera manager kafka configuration. and restart kafka service.


2)create jaas.conf file in your home path /home/userid/
vi jaas.conf
KafkaClient { required

#create new keytab by using principal of same user id and put it in below path

#below replace with your correct principal name


3)Create file containing the following properties in the same path /home/userid/
sudo vi


4) Login with same user id's login and do kinit

kinit -kt useridkerberos.keytab userid@REALMHOSTNAME.COM


5) Cretaing Topics:
/usr/bin/kafka-topics --create --zookeeper hostname1:2181,hostname2:2181,hostname3:2181/kafka --replication-factor 2 --partitions 2 --topic newtopic1


6) Describing Topics:

/usr/bin/kafka-topics --describe --zookeeper hostname1:2181,hostname2:2181,hostname3:2181/kafka --topic testtopic1


7)  export KAFKA_OPTS=""
verify it:  echo "$KAFKA_OPTS"


8) Writting message using Producer:
/usr/bin/kafka-console-producer --broker-list brokerhostname1:9092,brokerhostname2:9092 --topic newtopic6 --producer.config


9) Open duplicate session of smae machine and run consumer command

Reading message using Consumer:

export KAFKA_OPTS=""

/usr/bin/kafka-console-consumer --new-consumer --topic newtopic6 --from-beginning --bootstrap-server brokerhostname1:9092,brokerhostname2:9092 --consumer.config



try it. It may work at your end also!!




New Contributor
Posts: 4
Registered: ‎08-29-2018

Re: Console producer/consumer not working in kafka 0.10.2 with kerberos


Thank you, but that is +- the same set of tasks I did before.


I found how to make consumer work, it was necessary to add  parameter  --partition 0:

KAFKA_OPTS="" kafka-console-consumer --bootstrap-server ourHost:9092 --topic test --consumer.config /root/ --partition 0

 I cannot see all the messages comming into the topic, but at least some of them which fall into specified partition are printed. Which is enought for me to confirm that Kafka broker works. 

I found this hint here:


Our community is getting a little larger. And a lot better.

Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.