Support Questions
Find answers, ask questions, and share your expertise

Kafka producer fails to retrieve metadata

Highlighted

Kafka producer fails to retrieve metadata

After creating a topic with partitions=1 and replication_factor=1, the kafka-console-producer script fails.

Error:

[2016-10-13 08:56:57,154] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,171] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,172] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: fast-messages1 (kafka.producer.async.DefaultEventHandler) [2016-10-13 08:56:57,279] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,285] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,285] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: fast-messages1 (kafka.producer.async.DefaultEventHandler) [2016-10-13 08:56:57,390] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,393] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,393] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: fast-messages1 (kafka.producer.async.DefaultEventHandler) [2016-10-13 08:56:57,499] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,506] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,506] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: fast-messages1 (kafka.producer.async.DefaultEventHandler) [2016-10-13 08:56:57,610] WARN Error while fetching metadata [{TopicMetadata for topic fast-messages1 -> No partition metadata for topic fast-messages1 due to kafka.common.LeaderNotAvailableException}] for topic [fast-messages1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2016-10-13 08:56:57,611] ERROR Failed to send requests for topics fast-messages1 with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler) [2016-10-13 08:56:57,612] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread) kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:91) at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68) at scala.collection.immutable.Stream.foreach(Stream.scala:547) at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)

2 REPLIES 2
Highlighted

Re: Kafka producer fails to retrieve metadata

Using HDP-2.3.2.0-2950

kafka version 0.8.2.2.3

Highlighted

Re: Kafka producer fails to retrieve metadata

Rising Star

i had a similar issue on kafka 0.10.0 ( latest), resolving the hostname or fdqn to ipaddress or loopback address resolved it.