Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Kafka on HDP 2.6,2 doesn't consume messages from console producer

SOLVED Go to solution
Highlighted

Kafka on HDP 2.6,2 doesn't consume messages from console producer

New Contributor

I'm getting started with Kafka on HDP(single kafka broker). I created a test topic, which went successful. Next I used the console producer to write some test messages bin/kafka-console-producer.sh --broker-list <kafka-broker-fqdn>:9092 --topic test Then it went blank where I typed some test message and did ^C to get back. When I validated the same using consumer I see nothing.

bin/kafka-console-consumer.sh --bootstrap-server <samehost-fqdn>:9092 --topic test --from-beginning. I've also updated the advertised.host.name in server.properties to the host's fqdn. Is there anything I'm missing?

I tried the same from a stand-alone kafka-zookeeper insatlltion outside of HDP which worked fine.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Rising Star

@Swaapnika Guntaka

As listener port is set to 6667, this should be used in the bootstrap server in your producer code as well as if you are running console producer/consumer. Kindly run the producer with port 6667 and let us know.

The error in the server log is related to the metrics. Could you please provide complete log and also server.properties.

Thanks!

7 REPLIES 7

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Rising Star

@Swaapnika Guntaka

Could you please make sure what port is set in the listener property of Kafka? Also Make sure producer is producing messages or not. You can wait for a while after producing messages to see whether its timing out or not.

Thanks!

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

New Contributor

@dbains The listeners is using the default port 9092. The producer is not producing messages is what I observed. When I start the console producer it just goes blank showing nothing.

The only thing I see in the logs is this.

[2017-11-10 14:07:21,947] INFO [Group Metadata Manager on Broker 1001]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

New Contributor

1. Plz double check listener port default is 6667 in hdp 2.6. chance to 9092 and restart.

2. Try using zookeeper for consumer console.

It works for me though.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

@Swaapnika Guntaka : 9092 is not a default port in HDP rather it is 6667, Could you please check the port and re-run the producer.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

New Contributor

The listener port is 6667. and I'm using 9092 for the bootstrap server when I try to produce messages. The kafka server starts fine and give this error after a while.

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

Rising Star

@Swaapnika Guntaka

As listener port is set to 6667, this should be used in the bootstrap server in your producer code as well as if you are running console producer/consumer. Kindly run the producer with port 6667 and let us know.

The error in the server log is related to the metrics. Could you please provide complete log and also server.properties.

Thanks!

Re: Kafka on HDP 2.6,2 doesn't consume messages from console producer

New Contributor

Setting to 6667 worked. Thanks