Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka console producer/consumer failing with AD users but works with local kafka realm.

SOLVED Go to solution
Highlighted

Kafka console producer/consumer failing with AD users but works with local kafka realm.

Expert Contributor

Hi,

 

We have just built a new kafka cluster(3.1.0-1.3.1.0.p0.35) and integrated it with kerberos. Kerberos is integrated with AD. We are able to produce and consume with kafka principle which is local but with AD users, it fails with below error on console.

 

19/06/09 08:14:22 DEBUG authenticator.SaslClientAuthenticator: Set SASL client state to CLIENT_COMPLETE
19/06/09 08:14:22 DEBUG authenticator.SaslClientAuthenticator: Set SASL client state to COMPLETE
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Initiating API versions fetch from node -1.
19/06/09 08:14:22 DEBUG network.Selector: [Producer clientId=console-producer] Connection with server2.kafka4.corp/180.20.92.23 disconnected
java.io.EOFException
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:124)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:93)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:235)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:196)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:545)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:483)
at org.apache.kafka.common.network.Selector.poll(Selector.java:412)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:481)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
at java.lang.Thread.run(Thread.java:748)
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Node -1 disconnected.
19/06/09 08:14:22 INFO clients.NetworkClient: [Producer clientId=console-producer] API versions request failed via disconnect. Defaulting legacy API versions
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Give up sending metadata request since no node is available
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Give up sending metadata request since no node is available
^C19/06/09 08:14:22 INFO producer.KafkaProducer: [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
19/06/09 08:14:22 DEBUG internals.Sender: [Producer clientId=console-producer] Beginning shutdown of Kafka producer I/O thread, sending remaining records.

 

==========================================

 

The broker logs only have errors about the shortname for users.

 

Caused by: org.apache.kafka.common.security.kerberos.NoMatchingRule: No rules apply to RAJESH@KAFKA4.CORP, rules [DEFAULT]

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Kafka console producer/consumer failing with AD users but works with local kafka realm.

Expert Contributor

@satz 

 

We were able to resolve this. We had the kerberos auth principles in default kafka group while all the broker were in a different config group. Adding the auth principles to the kafka config group has solved the issue.

2 REPLIES 2

Re: Kafka console producer/consumer failing with AD users but works with local kafka realm.

Rising Star

Hello @RajeshBodolla 

 

It seems like the actual exception is as below

 

Caused by: org.apache.kafka.common.security.kerberos.NoMatchingRule: No rules apply to RAJESH@KAFKA4.CORP, rules [DEFAULT]

 

Could you please share the complete stack? it seems it is somewhere not able to resolve the principal

Thanks,
Satz

Re: Kafka console producer/consumer failing with AD users but works with local kafka realm.

Expert Contributor

@satz 

 

We were able to resolve this. We had the kerberos auth principles in default kafka group while all the broker were in a different config group. Adding the auth principles to the kafka config group has solved the issue.