My current setup has 1 management node and 3 worker node. Server A(master) - kafka broker, ... Server B - zookeeper, kafka broker, ... Server C - zookeeper, kafka broker, ... Server D - zookeeper, kafka broker, ...
I am using a client machine in the same network to create topic, produce and consume message using kafka bat files.
I realise when i create a topic, and the leader of the topic happens to be the broker on server A, everything works.
But if the leader of the topic is any broker on server b/c/d, when i attempt to send any messages, it prompt the following message. WARN - Got error produce response with correlation id _ on topic-partition ___, retrying. Error: UNKNOWN_TOPIC_OR_PARTITION WARN - Receive unknown topic or partition error in produce request on partition ___. The topic/partition may not exist or the user may not have Describe access to it ERROR - Error when sending message to topic ___ with key: null, value... UnknownTopicOrPartitionException: This server does not host this topic-partition.
I am suspecting something to do with the zookeeper, but have no idea where to look deeper into. I am a beginner in this area, any help would be appreciated.