Support Questions

Find answers, ask questions, and share your expertise

kafka consumer not showing the consumed messages ?


I am following and few weeks earlier everything was working well, but when I started it over again today is not working, I am using kafka_2.11-1.1.0.

I created the new topic 'rahul' with the following command :

bin/ --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic rahul
Created topic "rahul"

I also re-checked the topics with

bin/ --list --zookeeper localhost:2181


started the producer:

bin/ --broker-list localhost:9092 --topic rahul


But when the time comes to consumer to show the messages: there is nothing although the command seems to be executed, but no messages.

bin/ --bootstrap-server localhost:9092 --topic rahul --from-beginning

I even tried consuming both ways as zookeeper and bootstrap server, but none of them are giving messages published.



Hey @Rahul Kumar!
How much is set for log.retention.hours?
And could you check if your kafka-console-consumer are creating a consumer group?

[root@node1 ~]# --zookeeper $ZKENSEMBLE --list
[root@node1 ~]# --bootstrap-server node1:6667 --list
#If shows smtg, try to describe it with the --group <consumer-group-mynumber> and --describe
#And we can also check the offset of your topic.
root@node1 ~]# --zkconnect $ZKENSEMBLE --output-file zk_offset_kafka
[root@node1 ~]# --broker-list node1:6667 --topic vin-hcc-nifi --time -1 #latest
[root@node1 ~]# --broker-list node1:6667 --topic vin-hcc-nifi --time -2 #earliest 

Hope this helps!


Hi @Vinicius Higa Murakami

I removed zookeeper,kafka and their respective log directories.
I installed zookeeper and kafka again.

So for the first time messages were consumed and published too on the terminal interface.
After that I stopped kafka and restarted kafka again... from this point the message is not getting consumed.

Why this is happening ? This is very unusal from kafka 😞

Hey @Rahul Kumar.
Just asking, but after you stopped kafka/zookeeper, did you tried to produce and consume messages again?
For example, let's say that you just did a kafka-console-consumer after 7 days, probably you won't be able to see that messages again on that topic, because Kafka has a parameter that retains messages for a determined period of time, which is log.retention.hours = 168 hours (7 days) by default (you can change it).

But, if you did the whole process again (create a topic, kafka-console-producer and kafka-console-consumer) after the kafka cluster was down, then we may need to take a look at the errors from the logs of Kafka/ZK and watch the consumer groups/offsets.
Hope this helps!


Hello @Rahul Kumar. Could you check your /etc/hosts? I saw you are using "localhost" as your broker/bootstrap. Try to change it to your "hostname" instead of "localhost".

Some softwares doesn´t use "loopback" as a listener port by default.

Besides that, if you are using the native Kafka from HortonWorks, the default port for brokers is 6667 or 6668 (when using Kerberos) and not 9092.

I hope this works.