question Re: Problem kafka producer - WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker in Support Questions

Problem kafka producer - WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker

yagoaparecidoti — Fri, 08 Jul 2022 15:46:05 GMT

we are using kafka 2.2.1 in cloudera manager with cdh6.3.2

when we try to run kafka producer:

kafka-console-producer --broker-list ip-172-31-13-30.us-east-2.compute.internal:9092,ip-172-31-12-151.us-east-2.compute.internal:9092,ip-172-31-14-136.us-east-2.compute.internal:9092 --topic test_topic

returns the errors:

22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-14-136.us-east-2.compute.internal:9092 (id: -3 rack: null) disconnected
22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-12-151.us-east-2.compute.internal:9092 (id: -2 rack: null) disconnected
22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-13-30.us-east-2.compute.internal:9092 (id: -1 rack: null) disconnected

when we check the logs of kafka brokers there is:

2022-07-08 12:36:17,365 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=89] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.)

2022-07-08 12:34:11,776 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=88] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.)

2022-07-08 12:34:11,505 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=90] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.)

We would like to know why this problem is occurring?

ps: we are managing to list the topics without problem

Re: Problem kafka producer - WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker

bgoerlitz — Fri, 08 Jul 2022 18:31:50 GMT

These errors on the client side usually indicate a security misconfiguration, and based on the broker log messages you sent, I would guess that you have Kerberos enabled for this cluster ("SASL handshake" suggests kerberos is expected). When anything other than PLAINTEXT (no security) is used, you will need to provide a client configuration file specifying the security properties.

Since the specific values of those properties would be dependent on your environment, I'd recommend that you review steps 7 and following in the Kafka Security - Enabling Kerberos Authentication documentation. Here is a link to that section

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kafka_security.html#kafka_kerberos

I hope that helps!

Re: Problem kafka producer - WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker

DianaTorres — Mon, 11 Jul 2022 18:52:48 GMT

@yagoaparecidoti Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks