Support Questions

Find answers, ask questions, and share your expertise

Problem kafka producer - WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker

we are using kafka 2.2.1 in cloudera manager with cdh6.3.2

 

when we try to run kafka producer:

 

kafka-console-producer --broker-list ip-172-31-13-30.us-east-2.compute.internal:9092,ip-172-31-12-151.us-east-2.compute.internal:9092,ip-172-31-14-136.us-east-2.compute.internal:9092 --topic test_topic

 

returns the errors:

 

22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-14-136.us-east-2.compute.internal:9092 (id: -3 rack: null) disconnected
22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-12-151.us-east-2.compute.internal:9092 (id: -2 rack: null) disconnected
22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-13-30.us-east-2.compute.internal:9092 (id: -1 rack: null) disconnected

 

when we check the logs of kafka brokers there is:

 

2022-07-08 12:36:17,365 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=89] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.)

2022-07-08 12:34:11,776 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=88] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.)

2022-07-08 12:34:11,505 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=90] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.)

 

We would like to know why this problem is occurring?

 

ps: we are managing to list the topics without problem

1 ACCEPTED SOLUTION

Cloudera Employee

These errors on the client side usually indicate a security misconfiguration, and based on the broker log messages you sent, I would guess that you have Kerberos enabled for this cluster ("SASL handshake" suggests kerberos is expected). When anything other than PLAINTEXT (no security) is used, you will need to provide a client configuration file specifying the security properties.

Since the specific values of those properties would be dependent on your environment, I'd recommend that you review steps 7 and following in the Kafka Security - Enabling Kerberos Authentication documentation. Here is a link to that section

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kafka_security.html#kafka_kerberos

 

I hope that helps!

View solution in original post

2 REPLIES 2

Cloudera Employee

These errors on the client side usually indicate a security misconfiguration, and based on the broker log messages you sent, I would guess that you have Kerberos enabled for this cluster ("SASL handshake" suggests kerberos is expected). When anything other than PLAINTEXT (no security) is used, you will need to provide a client configuration file specifying the security properties.

Since the specific values of those properties would be dependent on your environment, I'd recommend that you review steps 7 and following in the Kafka Security - Enabling Kerberos Authentication documentation. Here is a link to that section

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kafka_security.html#kafka_kerberos

 

I hope that helps!

Community Manager

@yagoaparecidoti Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.