question Kafka on HDP 2.6,2 doesn't consume messages from console producer in Support Questions

Kafka on HDP 2.6,2 doesn't consume messages from console producer

guntaka — Thu, 09 Nov 2017 08:31:26 GMT

I'm getting started with Kafka on HDP(single kafka broker). I created a test topic, which went successful. Next I used the console producer to write some test messages bin/kafka-console-producer.sh --broker-list <kafka-broker-fqdn>:9092 --topic test Then it went blank where I typed some test message and did ^C to get back. When I validated the same using consumer I see nothing.

bin/kafka-console-consumer.sh --bootstrap-server <samehost-fqdn>:9092 --topic test --from-beginning. I've also updated the advertised.host.name in server.properties to the host's fqdn. Is there anything I'm missing?

I tried the same from a stand-alone kafka-zookeeper insatlltion outside of HDP which worked fine.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

dbains — Fri, 10 Nov 2017 02:32:01 GMT

@Swaapnika Guntaka

Could you please make sure what port is set in the listener property of Kafka? Also Make sure producer is producing messages or not. You can wait for a while after producing messages to see whether its timing out or not.

Thanks!

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

guntaka — Sat, 11 Nov 2017 07:12:28 GMT

@dbains The listeners is using the default port 9092. The producer is not producing messages is what I observed. When I start the console producer it just goes blank showing nothing.

The only thing I see in the logs is this.

[2017-11-10 14:07:21,947] INFO [Group Metadata Manager on Broker 1001]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

NikhilSilsarma — Sun, 12 Nov 2017 19:12:56 GMT

1. Plz double check listener port default is 6667 in hdp 2.6. chance to 9092 and restart.

2. Try using zookeeper for consumer console.

It works for me though.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

sandyy006 — Mon, 13 Nov 2017 01:48:43 GMT

@Swaapnika Guntaka : 9092 is not a default port in HDP rather it is 6667, Could you please check the port and re-run the producer.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

guntaka — Sun, 18 Aug 2019 07:11:11 GMT

The listener port is 6667. and I'm using 9092 for the bootstrap server when I try to produce messages. The kafka server starts fine and give this error after a while.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

dbains — Tue, 14 Nov 2017 03:45:33 GMT

@Swaapnika Guntaka

As listener port is set to 6667, this should be used in the bootstrap server in your producer code as well as if you are running console producer/consumer. Kindly run the producer with port 6667 and let us know.

The error in the server log is related to the metrics. Could you please provide complete log and also server.properties.

Thanks!

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

guntaka — Wed, 15 Nov 2017 01:25:08 GMT

Setting to 6667 worked. Thanks