Reply
Highlighted
Expert Contributor
Posts: 104
Registered: ‎10-04-2017
Accepted Solution

Kafka console producer/consumer failing with AD users but works with local kafka realm.

Hi,

 

We have just built a new kafka cluster(3.1.0-1.3.1.0.p0.35) and integrated it with kerberos. Kerberos is integrated with AD. We are able to produce and consume with kafka principle which is local but with AD users, it fails with below error on console.

 

19/06/09 08:14:22 DEBUG authenticator.SaslClientAuthenticator: Set SASL client state to CLIENT_COMPLETE
19/06/09 08:14:22 DEBUG authenticator.SaslClientAuthenticator: Set SASL client state to COMPLETE
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Initiating API versions fetch from node -1.
19/06/09 08:14:22 DEBUG network.Selector: [Producer clientId=console-producer] Connection with server2.kafka4.corp/180.20.92.23 disconnected
java.io.EOFException
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:124)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:93)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:235)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:196)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:545)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:483)
at org.apache.kafka.common.network.Selector.poll(Selector.java:412)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:481)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
at java.lang.Thread.run(Thread.java:748)
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Node -1 disconnected.
19/06/09 08:14:22 INFO clients.NetworkClient: [Producer clientId=console-producer] API versions request failed via disconnect. Defaulting legacy API versions
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Give up sending metadata request since no node is available
19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Give up sending metadata request since no node is available
^C19/06/09 08:14:22 INFO producer.KafkaProducer: [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
19/06/09 08:14:22 DEBUG internals.Sender: [Producer clientId=console-producer] Beginning shutdown of Kafka producer I/O thread, sending remaining records.

 

==========================================

 

The broker logs only have errors about the shortname for users.

 

Caused by: org.apache.kafka.common.security.kerberos.NoMatchingRule: No rules apply to RAJESH@KAFKA4.CORP, rules [DEFAULT]

Posts: 60
Topics: 0
Kudos: 8
Solutions: 5
Registered: ‎05-15-2018

Re: Kafka console producer/consumer failing with AD users but works with local kafka realm.

Hello @RajeshBodolla 

 

It seems like the actual exception is as below

 

Caused by: org.apache.kafka.common.security.kerberos.NoMatchingRule: No rules apply to RAJESH@KAFKA4.CORP, rules [DEFAULT]

 

Could you please share the complete stack? it seems it is somewhere not able to resolve the principal

Thanks,
Satz
Expert Contributor
Posts: 104
Registered: ‎10-04-2017

Re: Kafka console producer/consumer failing with AD users but works with local kafka realm.

@satz 

 

We were able to resolve this. We had the kerberos auth principles in default kafka group while all the broker were in a different config group. Adding the auth principles to the kafka config group has solved the issue.