question Kafka 0.9 new-producer on kerberized HDP 2.4 in Archives of Support Questions (Read Only)

Kafka 0.9 new-producer on kerberized HDP 2.4

saumidh_mhatre — Mon, 20 Jun 2016 14:50:39 GMT

I am trying to run kafka-console-producer.sh script on a kerberized HDP 2.4 sandbox with Kafka 0.9. I have created a topic and provided user permissions to the topic.

Below is the command I am using to start the producer:

./kafka-console-producer.sh --topic test-topic --broker-list sandbox.hortonworks.com:6667 --security-protocol PLAINTEXTSASL --new-producer

Below is the error I get when I try to produce something:

ERROR Error when sending message to topic test-topic with key: null, value: 8 bytes with error: Failed to update metadata after 60000 ms. (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)

It works fine when I don't use the new-producer, but I need to use the new producer API.

Re: Kafka 0.9 new-producer on kerberized HDP 2.4

rreddy — Tue, 21 Jun 2016 01:11:49 GMT

For new producer api, try increasing the metadata.fetch.timeout.ms in producer config and also socket.timeout.ms

Re: Kafka 0.9 new-producer on kerberized HDP 2.4

saumidh_mhatre — Tue, 21 Jun 2016 18:15:15 GMT

I changed the above properties to 600000 ms, still the issue exists.

Re: Kafka 0.9 new-producer on kerberized HDP 2.4

rreddy — Wed, 22 Jun 2016 07:07:27 GMT

What is your config/server.properties and config/producer.properties? Anything in zookeeper logs?

Re: Kafka 0.9 new-producer on kerberized HDP 2.4

pavanireddy525 — Thu, 30 Jun 2016 01:34:09 GMT

We had same issue and increasing time out values worked.