In a PoC cluster provided with kafka(1 broker) I'm trying to send messages to the cluster with an external producer through a public IP. My cluster nodes are hosted in Azure and all of them are reachable from my local machine where the external producer is running.
Following this great article https://rmoff.net/2018/08/02/kafka-listeners-explained/I figured out the kafka listeners config I need as this broker public IP is only reachable from my local machine and inside the cluster they can only communicate using their internal hostnames:
listeners=INTERNAL://0.0.0.0:19092,EXTERNAL://0.0.0.0:9092 listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT advertised.listeners=INTERNAL://internal-broker-node-hostname:19092,EXTERNAL://public-broker-ip:9092 inter.broker.listener.name=INTERNAL
With this configuration when I try to start the broker I get this error:
org.apache.kafka.common.config.ConfigException: Only one of inter.broker.listener.name and security.inter.broker.protocol should be set.
Which makes sense as in Kafka doc for this property inter.broker.listener.name they say:
Name of listener used for communication between brokers. If this is unset, the listener name is defined by security.inter.broker.protocol. It is an error to set this and security.inter.broker.protocol properties at the same time.
The problem I'm facing is that I'm unable to unset security.inter.broker.protocol through the cloudera manager as I only have the option of switching options in a radio button.
How can I unset that property?
Hi I know this is super late, but I was able to resolve this by doing this:
listeners=PLAINTEXT://0.0.0.0:19092,EXTERNAL://0.0.0.0:9092 listener.security.protocol.map=PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT advertised.listeners=PLAINTEXT://internal-broker-node-hostname:19092,EXTERNAL://public-broker-ip:9092
I also changed the security.inter.broker.protocol to PLAINTEXT