[kafka] Error sending message when leader is not master node

Qing — Fri, 16 Sep 2022 13:22:55 GMT

Hi,

My current setup has 1 management node and 3 worker node.
Server A(master) - kafka broker, ...
Server B - zookeeper, kafka broker, ...
Server C - zookeeper, kafka broker, ...
Server D - zookeeper, kafka broker, ...

KAFKA-3.0.0-1.3.0.0.p0.40
CDH-5.14.2-1.cdh5.14.2.p0.3

I am using a client machine in the same network to create topic, produce and consume message using kafka bat files.

I realise when i create a topic, and the leader of the topic happens to be the broker on server A, everything works.

But if the leader of the topic is any broker on server b/c/d, when i attempt to send any messages, it prompt the following message.
WARN - Got error produce response with correlation id _ on topic-partition ___, retrying. Error: UNKNOWN_TOPIC_OR_PARTITION
WARN - Receive unknown topic or partition error in produce request on partition ___. The topic/partition may not exist or the user may not have Describe access to it
ERROR - Error when sending message to topic ___ with key: null, value... UnknownTopicOrPartitionException: This server does not host this topic-partition.

I am suspecting something to do with the zookeeper, but have no idea where to look deeper into.
I am a beginner in this area, any help would be appreciated.

Thanks in advance!

Re: [kafka] Error sending message when leader is not master node

Qing — Wed, 27 Jun 2018 07:37:50 GMT

Manage to get it to work after putting broker and zookeeper on different node.
Not sure what is the reason, if anyone know please advise. Thanks!

Re: [kafka] Error sending message when leader is not master node

Qing — Wed, 27 Jun 2018 08:05:53 GMT

Apologies, kafka broker can be same node as the zookeeper. Tested ok.

My issue was due to dns
Host file not set properly.
Pardon me.

question [kafka] Error sending message when leader is not master node in Archives of Support Questions (Read Only)

[kafka] Error sending message when leader is not master node

Re: [kafka] Error sending message when leader is not master node

Re: [kafka] Error sending message when leader is not master node