Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

wrong alerts when configuring two listeners for Kafka in Ambari

Solved Go to solution

wrong alerts when configuring two listeners for Kafka in Ambari

Explorer

I install Kafka in Ambari and configure two listeners for every broker, one for inner traffic and one for outer traffic. E.g., broker A has two ip addresses: 192.168.1.1 and 88.88.88.88, and its configuration is like follows:

 

SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092
listener.security.protocol.map=SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
security.inter.broker.protocol=SASL_PLAINTEXT

 

With this configurations, Kafka works fine but keeps alerting, the message is:

 

Connection failed: [Errno 111] Connection refused to 192.168.1.1:19092

 

 This is not valid because ip and port do not match. And I suspect that it could just split by colon and use the last element. So I exchange the order of the two listeners:

 

EXTERNAL://88.88.88.88:19092,SASL_PLAINTEXT://192.168.1.1:9092

 

To my surprise, it still alerts and the message changes to 

 

Connection failed: [Errno 111] Connection refused to 88.88.88.88:9092

 

ip and port still do not match ? 
Have anybody also run into this or am I misconfiguring something ? Or this is a bug ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: wrong alerts when configuring two listeners for Kafka in Ambari

Explorer

@Shelton Glad that you reproduce this problem. I have ten brokers and every broker is configured with their respective IP addresses but I am afraid I cannot provide screenshot for some reasons. I have a workaround for this problem. In my own opinion, the reason of this problem is that Ambari cannot recognize IP and port bindings correctly. So to solve this is to avoiding the challenge for it. Here is my configuration now:

 

listeners=SASL_PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:19092
advertised.listeners=SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092

The other configurations stay unchanged. Notice that I change `listeners` so that even Ambari  cannot recognize the right IP and port binding, both ports are now available on all interfaces so there are no false alerts now.

Thanks you so much for helping me on this problem and apologize for this late response.

5 REPLIES 5

Re: wrong alerts when configuring two listeners for Kafka in Ambari

Mentor

@iamabug 

I think it's a misconfiguration can you see the differences between these 2? The one in Black and BOLD is your current remove the ( ) and replace it with a colon ( : ) and space after 

 

listener.security.protocol.map=SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT          [old]


listener.security.protocol.map: SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT              [New]

Restart the brokers and let me know if you still encounter the problem

 

Re: wrong alerts when configuring two listeners for Kafka in Ambari

Explorer

@Shelton Hi, I am only using (=) to represent the configuration key and value. In Ambari, there is no need and no place to type (=) or (: ).

Re: wrong alerts when configuring two listeners for Kafka in Ambari

Mentor

@iamabug 

I now get you so yours is Ambari managed let me get back to you after testing

Highlighted

Re: wrong alerts when configuring two listeners for Kafka in Ambari

Mentor

@iamabug 

I think I have seen the problem, can you share your Ambari UI screenshot of your configured listeners? Or where have you declared these parameters

SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092
listener.security.protocol.map=SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
security.inter.broker.protocol=SASL_PLAINTEXT

Only one broker? I am sorry I can only help in the evenings I have a full time job that consumes my time

 

 

 

 

Re: wrong alerts when configuring two listeners for Kafka in Ambari

Explorer

@Shelton Glad that you reproduce this problem. I have ten brokers and every broker is configured with their respective IP addresses but I am afraid I cannot provide screenshot for some reasons. I have a workaround for this problem. In my own opinion, the reason of this problem is that Ambari cannot recognize IP and port bindings correctly. So to solve this is to avoiding the challenge for it. Here is my configuration now:

 

listeners=SASL_PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:19092
advertised.listeners=SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092

The other configurations stay unchanged. Notice that I change `listeners` so that even Ambari  cannot recognize the right IP and port binding, both ports are now available on all interfaces so there are no false alerts now.

Thanks you so much for helping me on this problem and apologize for this late response.

Don't have an account?
Coming from Hortonworks? Activate your account here