Member since
08-19-2019
25
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 5105 | 11-09-2019 05:07 PM |
11-09-2019
05:07 PM
@Shelton Glad that you reproduce this problem. I have ten brokers and every broker is configured with their respective IP addresses but I am afraid I cannot provide screenshot for some reasons. I have a workaround for this problem. In my own opinion, the reason of this problem is that Ambari cannot recognize IP and port bindings correctly. So to solve this is to avoiding the challenge for it. Here is my configuration now: listeners=SASL_PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:19092
advertised.listeners=SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092 The other configurations stay unchanged. Notice that I change `listeners` so that even Ambari cannot recognize the right IP and port binding, both ports are now available on all interfaces so there are no false alerts now. Thanks you so much for helping me on this problem and apologize for this late response.
... View more
11-05-2019
05:36 PM
@Shelton Hi, I am only using (=) to represent the configuration key and value. In Ambari, there is no need and no place to type (=) or (: ).
... View more
11-05-2019
03:19 AM
I install Kafka in Ambari and configure two listeners for every broker, one for inner traffic and one for outer traffic. E.g., broker A has two ip addresses: 192.168.1.1 and 88.88.88.88, and its configuration is like follows:
SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092
listener.security.protocol.map=SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
security.inter.broker.protocol=SASL_PLAINTEXT
With this configurations, Kafka works fine but keeps alerting, the message is:
Connection failed: [Errno 111] Connection refused to 192.168.1.1:19092
This is not valid because ip and port do not match. And I suspect that it could just split by colon and use the last element. So I exchange the order of the two listeners:
EXTERNAL://88.88.88.88:19092,SASL_PLAINTEXT://192.168.1.1:9092
To my surprise, it still alerts and the message changes to
Connection failed: [Errno 111] Connection refused to 88.88.88.88:9092
ip and port still do not match ? Have anybody also run into this or am I misconfiguring something ? Or this is a bug ?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Kafka
08-29-2019
04:47 AM
The reason why I am asking this is a little complicated. So I will not mention too much about it (unless somebody is interested). In fact, I have more than one question.
1. When Kerberos is enabled, does CM add service principals based on hostname shown/stored in CM or every host's actual hostname ? Otherwise, are they always the same ?
2. If hostnames in CM can be different from actual hostnames, how do I achieve that ? (I have tried changing database records and the host cannot connect to HostMonitor afterwards)
Thanks.
... View more
Labels:
- Labels:
-
Cloudera Manager
08-26-2019
05:07 AM
It's really nice of you. I would definitely ask for your help when something tricky comes up. Thank you very much.
... View more
08-26-2019
12:50 AM
Thanks for your answer. I now believe that ACL in Zookeeper may be the solution here.
... View more
08-22-2019
11:30 PM
I have enabled Kerberos authentication for Kafka as the documentation suggests and indeed producing to topics and consuming from topics requires authentication. Surprisingly, topic creation and deletion do not require authentication. Could somebody tell me whether this goes wrong ? Really appreciate it.
CDH version: 5.15.1
CDK version: 4.1.0
test command:
kafka-topics --create --zookeeper <zookeeper-host>:2181 --replication-factor 2 --partitions 3 --topic test2
kafka-topics --delete --zookeeper <zookeeper-host>:2181 --topic test2
... View more
Labels:
- Labels:
-
Apache Kafka
08-21-2019
06:10 AM
Now I am really clear about the situation. Thanks a lot for your replies.
... View more
08-21-2019
06:04 AM
1 Kudo
I have CDK 3.1.0 running on cluster 1 and CDK 4.1.0 running on cluster 2. And CDH version is 5.15.1.
... View more