Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

kafka listener address

kafka listener address

Rising Star

Hello,

I have a hadoop cluster which contain multi network interface nodes. One interface is for public reaches other for cluster internal. Kafka broker was listening internal ip as default but if a server is outside cluster it couldn't reach, couldn't telnet to server.(something like nifi)

I changed listener adress to 0.0.0.0 and restarted kafka. This time can telnet to kafka broker server but when I try to produce a message it fails with this errors(even producer is executed inside cluster);

[2017-08-04 17:18:05,688] ERROR Error when sending message to topic test with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
	org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test-0 due to 1515 ms has passed since batch creation plus linger time
8 REPLIES 8
Highlighted

Re: kafka listener address

which version of kafka you are using (does it use zookeeper)?, Can you enable DEBUG level and try to find the real port which it is trying to connect..

is your kafka secure(using SSL or SASL)?

Re: kafka listener address

Rising Star

it is 0.10.0.2.6(which comes with hdp 2.6, it uses zookeeper). It uses port 6667. It is not secure.

Re: kafka listener address

Did you try telnet on zookeeper port as well?

Re: kafka listener address

Expert Contributor

How do you test to produce messages? Is it custom code or did you try first with the producer example script that come with the installation?

Run the producer and then type a few messages into the console to send to the server.

> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
This is a message
This is another message

Does it work?

Re: kafka listener address

Rising Star

yes I tried to produce like this.

Re: kafka listener address

Expert Contributor

Can you check if you have the correct port? normally it's configured with port 6667 for the broker.

Can you also copy/paste the command that you use to start the producer?

Re: kafka listener address

Expert Contributor

@Mustafa Kemal MAYUK did you success to solve this? if yes can provide your answer or accept an answer?

Re: kafka listener address

Rising Star

I resolved by adding a custom kafka-broker variable, advertised.listeners. It is fqdn of broker host. Also changed listener address to 0.0.0.0, it worked.

Don't have an account?
Coming from Hortonworks? Activate your account here