question Configuring two listeners in Kerberized Kafka in Support Questions

question Configuring two listeners in Kerberized Kafka in Support Questions https://community.cloudera.com/t5/Support-Questions/Configuring-two-listeners-in-Kerberized-Kafka/m-p/269091#M206624 I have configured two listeners for non-kerberized Kafka before and it works out fine. That configuration for every broker is as follows:     <LI-CODE lang="markup">security.inter.broker.protocol=PLAINTEXT listeners=PLAINTEXT://<internal-ip>:9092,EXTERNAL://<external-ip>:19092 listener.security.protocol.map=PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT</LI-CODE>     I checked the actual configuration file kafka.properties used by running Kafka process and it is as follows:     <LI-CODE lang="markup">listeners=PLAINTEXT://<broker-hostname>:9092, ... ... listeners=PLAINTEXT://<internal-ip>:9092,EXTERNAL://<external-ip>:19092 listener.security.protocol.map=PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT</LI-CODE>     Though it includes two same entries for listeners, while I am not sure whether this is a bug or not, our Kafka works as expected.  The problem is I use the similar configuration for new Kerberized Kafka but it cannot work. The configuration is as follows:     <LI-CODE lang="markup">security.inter.broker.protocol=SASL_PLAINTEXT listeners=SASL_PLAINTEXT://<broker-hostname>:9092, ... ... listeners=SASL_PLAINTEXT://<internal-ip>:9092,EXTERNAL://<external-ip>:19092 listener.security.protocol.map=SASL_PLAINTEXT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT</LI-CODE>     I believe the configuration does not work based on that when I comment all broker hostnames in /etc/hosts kafka-console-consumer/producer scripts raise error:     <LI-CODE lang="markup">9/08/28 10:32:01 ERROR clients.NetworkClient: [Producer clientId=console-producer] Connection to node -1 failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. 19/08/28 10:32:01 ERROR internals.ErrorLoggingCallback: Error when sending message to topic test1 with key: null, value: 3 bytes with error: org.apache.kafka.common.errors.SaslAuthenticationException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:361) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:359) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:359) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslClientToken(SaslClientAuthenticator.java:269) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:206) at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:81) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:474) at org.apache.kafka.common.network.Selector.poll(Selector.java:412) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:481) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163) at java.lang.Thread.run(Thread.java:748) Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 14 more Caused by: KrbException: Server not found in Kerberos database (7) - LOOKING_UP_SERVER at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:251) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:262) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:308) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:126) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693) ... 17 more</LI-CODE>     Does anybody else run into this ? I could really use some help. Thanks. Wed, 28 Aug 2019 13:57:29 GMT iamabug 2019-08-28T13:57:29Z