We ran into the same issue because we rely on poor mans DNS via local hosts file, as we don't have control over the infrastructure.
To solve this issue of advertising non existent hostnames there are two solutions:
1. create separate configuration groups for each kafka broker and override `listeners` property with explicit IP of the relevant node
2. setup ambari-agent to specify a custom public hostname and use template variable at kafka config to use that property
Solution 2 introduces a fix or another problem, it depends on your setup: By setting up ambari-agent to use a custom public hostname links from ambari to services like HDFS UI, YARN UI, Spark UI, Zeppelin, etc. will use this value.
To setup solution 2: Create public hostname script
Place a file at
/var/lib/ambari-agent/public_hostname.sh and make it executable chmod a+x /var/lib/ambari-agent/public_hostname.sh with following content:
#!/bin/sh
hostname -I | awk '{print $1}'
Change ambari-agent config
at
/etc/ambari-agent/conf/ambari-agent.ini , add property at agent section:
public_hostname_script=/var/lib/ambari-agent/public_hostname.sh
Restart ambari agent
ambari-agent restart
Configure Kafka Broker listener
PLAINTEXT://{{config['agentLevelParams']['public_hostname']}}:6667 Restart Kafka
... View more
