Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to set kafka to broadcast the ip address rather than host name?

avatar
Rising Star

I use kafka with default value(PLAINTEXT://localhost:9092), however, it broadcasts it's hostname instead of ip address. The result is, if the producer runs a host without the kafka host info, it cannot send messages to kafka. How to resolve it?

1 REPLY 1

avatar

We ran into the same issue because we rely on poor mans DNS via local hosts file, as we don't have control over the infrastructure.

To solve this issue of advertising non existent hostnames there are two solutions:
1. create separate configuration groups for each kafka broker and override `listeners` property with explicit IP of the relevant node
2. setup ambari-agent to specify a custom public hostname and use template variable at kafka config to use that property

Solution 2 introduces a fix or another problem, it depends on your setup: By setting up ambari-agent to use a custom public hostname links from ambari to services like HDFS UI, YARN UI, Spark UI, Zeppelin, etc. will use this value.

To setup solution 2:

Create public hostname script

Place a file at /var/lib/ambari-agent/public_hostname.sh and make it executable chmod a+x /var/lib/ambari-agent/public_hostname.sh with following content:

#!/bin/sh
hostname -I | awk '{print $1}'
Change ambari-agent config

at /etc/ambari-agent/conf/ambari-agent.ini, add property at agent section:

public_hostname_script=/var/lib/ambari-agent/public_hostname.sh
Restart ambari agent
ambari-agent restart
Configure Kafka Broker listener
PLAINTEXT://{{config['agentLevelParams']['public_hostname']}}:6667
Restart Kafka