Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.
iamabug
Contributor
My Accepted Solutions
Show All

Re: wrong alerts when configuring two listeners fo...

by Contributor in Support Questions
‎11-09-2019 05:07 PM
‎11-09-2019 05:07 PM
@Shelton Glad that you reproduce this problem. I have ten brokers and every broker is configured with their respective IP addresses but I am afraid I cannot provide screenshot for some reasons. I have a workaround for this problem. In my own opinion, the reason of this problem is that Ambari cannot recognize IP and port bindings correctly. So to solve this is to avoiding the challenge for it. Here is my configuration now:   listeners=SASL_PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:19092 advertised.listeners=SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092 The other configurations stay unchanged. Notice that I change `listeners` so that even Ambari  cannot recognize the right IP and port binding, both ports are now available on all interfaces so there are no false alerts now. Thanks you so much for helping me on this problem and apologize for this late response. ... View more

Re: wrong alerts when configuring two listeners fo...

by Contributor in Support Questions
‎11-05-2019 05:36 PM
‎11-05-2019 05:36 PM
@Shelton Hi, I am only using (=) to represent the configuration key and value. In Ambari, there is no need and no place to type (=) or (: ). ... View more

wrong alerts when configuring two listeners for Ka...

by Contributor in Support Questions
‎11-05-2019 03:19 AM
‎11-05-2019 03:19 AM
I install Kafka in Ambari and configure two listeners for every broker, one for inner traffic and one for outer traffic. E.g., broker A has two ip addresses: 192.168.1.1 and 88.88.88.88, and its configuration is like follows:   SASL_PLAINTEXT://192.168.1.1:9092,EXTERNAL://88.88.88.88:19092 listener.security.protocol.map=SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT security.inter.broker.protocol=SASL_PLAINTEXT   With this configurations, Kafka works fine but keeps alerting, the message is:   Connection failed: [Errno 111] Connection refused to 192.168.1.1:19092    This is not valid because ip and port do not match. And I suspect that it could just split by colon and use the last element. So I exchange the order of the two listeners:   EXTERNAL://88.88.88.88:19092,SASL_PLAINTEXT://192.168.1.1:9092   To my surprise, it still alerts and the message changes to    Connection failed: [Errno 111] Connection refused to 88.88.88.88:9092   ip and port still do not match ?  Have anybody also run into this or am I misconfiguring something ? Or this is a bug ? ... View more
Labels:

Re: Kafka: security.inter.broker.protocol can not ...

by Contributor in Support Questions
‎11-05-2019 03:06 AM
‎11-05-2019 03:06 AM
I think CDH only support Kerberos for SASL. I switch to Ambari to manager kafka and use SASL/PLAIN at last. ... View more

Re: Configuring two listeners in Kerberized Kafka

by Contributor in Support Questions
‎09-09-2019 06:58 PM
‎09-09-2019 06:58 PM
Sorry @ManuelCalvo , I probably have no time to do that because the test environment has been torn down. And I kind of have a theory for that. I guess that Kerberos principals are based on hostnames while I have changed Kafka listeners to IP addresses. With that config, Kafka clients cannot build a qualified principal name because they do not know hostnames of brokers. Thanks for you help anyway ! ... View more

Re: Kafka: security.inter.broker.protocol can not ...

by Contributor in Support Questions
‎09-04-2019 07:15 PM
‎09-04-2019 07:15 PM
@Denys Are you able to enable SASL/SCRAM for Kafka in CDH at last ? Thanks. ... View more

Re: Kafka: security.inter.broker.protocol can not ...

by Contributor in Support Questions
‎09-03-2019 07:15 PM
‎09-03-2019 07:15 PM
@AKR  I run into the same problem. I don't think SASL itself is restricted to use Kerberos protocol, according to wikipedia. As far as I am concerned, it seems that Cloudera recommends its users to use Kerberos and kind of forbids us using other mechanisms. I wonder the reason why we cannot or should not use other SASL mechanisms. ... View more

How do I change hostname shown in Cloudera Manager...

by Contributor in Support Questions
‎08-29-2019 04:47 AM
‎08-29-2019 04:47 AM
The reason why I am asking this is a little complicated. So I will not mention too much about it (unless somebody is interested). In fact, I have more than one question.   1. When Kerberos is enabled, does CM add service principals based on hostname shown/stored in CM or every host's actual hostname ? Otherwise, are they always the same ? 2. If hostnames in CM can be different from actual hostnames, how do I achieve that ? (I have tried changing database records and the host cannot connect to HostMonitor afterwards)   Thanks. ... View more
Labels:

Configuring two listeners in Kerberized Kafka

by Contributor in Support Questions
‎08-27-2019 07:46 PM
‎08-27-2019 07:46 PM
I have configured two listeners for non-kerberized Kafka before and it works out fine. That configuration for every broker is as follows:     security.inter.broker.protocol=PLAINTEXT listeners=PLAINTEXT://<internal-ip>:9092,EXTERNAL://<external-ip>:19092 listener.security.protocol.map=PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT     I checked the actual configuration file kafka.properties used by running Kafka process and it is as follows:     listeners=PLAINTEXT://<broker-hostname>:9092, ... ... listeners=PLAINTEXT://<internal-ip>:9092,EXTERNAL://<external-ip>:19092 listener.security.protocol.map=PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT     Though it includes two same entries for listeners, while I am not sure whether this is a bug or not, our Kafka works as expected.  The problem is I use the similar configuration for new Kerberized Kafka but it cannot work. The configuration is as follows:     security.inter.broker.protocol=SASL_PLAINTEXT listeners=SASL_PLAINTEXT://<broker-hostname>:9092, ... ... listeners=SASL_PLAINTEXT://<internal-ip>:9092,EXTERNAL://<external-ip>:19092 listener.security.protocol.map=SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT     I believe the configuration does not work based on that when I comment all broker hostnames in /etc/hosts kafka-console-consumer/producer scripts raise error:     9/08/28 10:32:01 ERROR clients.NetworkClient: [Producer clientId=console-producer] Connection to node -1 failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. 19/08/28 10:32:01 ERROR internals.ErrorLoggingCallback: Error when sending message to topic test1 with key: null, value: 3 bytes with error: org.apache.kafka.common.errors.SaslAuthenticationException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:361) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:359) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:359) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslClientToken(SaslClientAuthenticator.java:269) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:206) at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:81) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:474) at org.apache.kafka.common.network.Selector.poll(Selector.java:412) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:481) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163) at java.lang.Thread.run(Thread.java:748) Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 14 more Caused by: KrbException: Server not found in Kerberos database (7) - LOOKING_UP_SERVER at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:251) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:262) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:308) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:126) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693) ... 17 more     Does anybody else run into this ? I could really use some help. Thanks. ... View more
Labels:

Re: Topic creation and deletion are not protected ...

by Contributor in Support Questions
‎08-26-2019 05:07 AM
‎08-26-2019 05:07 AM
It's really nice of you. I would definitely ask for your help when something tricky comes up. Thank you very much. ... View more
Contact Me
Online
Offline
Last Visited
‎02-28-2021 08:48 PM
Public Statistics
Date Registered ‎08-19-2019 06:15 PM
Last Visited ‎02-28-2021 08:48 PM
Total Messages Posted 25
Total Kudos Received 1