Support Questions
Find answers, ask questions, and share your expertise

Kafka on HDP 2.6,2 doesn't consume messages from console producer

Solved Go to solution

Kafka on HDP 2.6,2 doesn't consume messages from console producer

Explorer

I'm getting started with Kafka on HDP(single kafka broker). I created a test topic, which went successful. Next I used the console producer to write some test messages bin/kafka-console-producer.sh --broker-list <kafka-broker-fqdn>:9092 --topic test Then it went blank where I typed some test message and did ^C to get back. When I validated the same using consumer I see nothing.

bin/kafka-console-consumer.sh --bootstrap-server <samehost-fqdn>:9092 --topic test --from-beginning. I've also updated the advertised.host.name in server.properties to the host's fqdn. Is there anything I'm missing?

I tried the same from a stand-alone kafka-zookeeper insatlltion outside of HDP which worked fine.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Rising Star

@Swaapnika Guntaka

As listener port is set to 6667, this should be used in the bootstrap server in your producer code as well as if you are running console producer/consumer. Kindly run the producer with port 6667 and let us know.

The error in the server log is related to the metrics. Could you please provide complete log and also server.properties.

Thanks!

View solution in original post

7 REPLIES 7

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Rising Star

@Swaapnika Guntaka

Could you please make sure what port is set in the listener property of Kafka? Also Make sure producer is producing messages or not. You can wait for a while after producing messages to see whether its timing out or not.

Thanks!

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Explorer

@dbains The listeners is using the default port 9092. The producer is not producing messages is what I observed. When I start the console producer it just goes blank showing nothing.

The only thing I see in the logs is this.

[2017-11-10 14:07:21,947] INFO [Group Metadata Manager on Broker 1001]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

1. Plz double check listener port default is 6667 in hdp 2.6. chance to 9092 and restart.

2. Try using zookeeper for consumer console.

It works for me though.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

@Swaapnika Guntaka : 9092 is not a default port in HDP rather it is 6667, Could you please check the port and re-run the producer.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Explorer

The listener port is 6667. and I'm using 9092 for the bootstrap server when I try to produce messages. The kafka server starts fine and give this error after a while.

42570-kafka-server-err.png

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Rising Star

@Swaapnika Guntaka

As listener port is set to 6667, this should be used in the bootstrap server in your producer code as well as if you are running console producer/consumer. Kindly run the producer with port 6667 and let us know.

The error in the server log is related to the metrics. Could you please provide complete log and also server.properties.

Thanks!

View solution in original post

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Explorer

Setting to 6667 worked. Thanks