Kafka Consumer is not working in CDH5.13.0 Kafka version : 0.10.2-kafka-2.2.0

[#1]

Hi,

I've been getting problems with the consumer in CDH5.13.0. For other CDH-5.11.0 and CDH-5.12.0 versions with Kafka version : 0.10.2-kafka-2.2.0 are working fine. Added advertised.listeners also and i'm running in a non-secure environment. Observed client.id is blank for console consumer whereas it is client.id = console-producer for producer. I need to work with CDH-5.13.0,please help me out

When I run the old API consumer, it works by running

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

However, when I run the new API consumer, I don't get anything when I run

bin/kafka-console-consumer.sh --new-consumer --topic test --from-beginning  --bootstrap-server localhost:9092

[#2 · Accepted solution]

Resolved by deleting brokers data from zookeeper
Thanks.

[#3]

How you deleted broker data from zookeeper??

I am having similar issue, with old --zookeeper, consumer is working fine however --bootstrap-server it is not working.

Kindly help?

[#4]

bin/zookeeper-shell.sh    <zookeeper_host:port>:2181

If you want to delete only topics

rmr /brokers/topics

for deleting brokers 

rmr /brokers

[#5]

I am using CDH 5.13 and CDK 4.0 (apache 2.1). Getting the same problem. However, consumer is still not getting messages after deleting brokers and topics as suggested by you. 

[#6]

Check if

offsets.topic.replication.factor

(or probably other config parameters related to replication) is not higher than the number of brokers. That was the problem in my case.