Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Kafka on HDP 2.6,2 doesn't consume messages from console producer

avatar
Explorer

I'm getting started with Kafka on HDP(single kafka broker). I created a test topic, which went successful. Next I used the console producer to write some test messages bin/kafka-console-producer.sh --broker-list <kafka-broker-fqdn>:9092 --topic test Then it went blank where I typed some test message and did ^C to get back. When I validated the same using consumer I see nothing.

bin/kafka-console-consumer.sh --bootstrap-server <samehost-fqdn>:9092 --topic test --from-beginning. I've also updated the advertised.host.name in server.properties to the host's fqdn. Is there anything I'm missing?

I tried the same from a stand-alone kafka-zookeeper insatlltion outside of HDP which worked fine.

1 ACCEPTED SOLUTION

avatar
Expert Contributor

@Swaapnika Guntaka

As listener port is set to 6667, this should be used in the bootstrap server in your producer code as well as if you are running console producer/consumer. Kindly run the producer with port 6667 and let us know.

The error in the server log is related to the metrics. Could you please provide complete log and also server.properties.

Thanks!

View solution in original post

7 REPLIES 7

avatar
Expert Contributor

@Swaapnika Guntaka

Could you please make sure what port is set in the listener property of Kafka? Also Make sure producer is producing messages or not. You can wait for a while after producing messages to see whether its timing out or not.

Thanks!

avatar
Explorer

@dbains The listeners is using the default port 9092. The producer is not producing messages is what I observed. When I start the console producer it just goes blank showing nothing.

The only thing I see in the logs is this.

[2017-11-10 14:07:21,947] INFO [Group Metadata Manager on Broker 1001]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)

avatar
Contributor

1. Plz double check listener port default is 6667 in hdp 2.6. chance to 9092 and restart.

2. Try using zookeeper for consumer console.

It works for me though.

avatar

@Swaapnika Guntaka : 9092 is not a default port in HDP rather it is 6667, Could you please check the port and re-run the producer.

avatar
Explorer

The listener port is 6667. and I'm using 9092 for the bootstrap server when I try to produce messages. The kafka server starts fine and give this error after a while.

42570-kafka-server-err.png

avatar
Expert Contributor

@Swaapnika Guntaka

As listener port is set to 6667, this should be used in the bootstrap server in your producer code as well as if you are running console producer/consumer. Kindly run the producer with port 6667 and let us know.

The error in the server log is related to the metrics. Could you please provide complete log and also server.properties.

Thanks!

avatar
Explorer

Setting to 6667 worked. Thanks