Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Apache Kafka partition reassignment reg.

Highlighted

Apache Kafka partition reassignment reg.

New Contributor

I have a Kafka cluster of 3 nodes. I want to add 2 more nodes to the cluster to make it 5 node cluster.
After adding the nodes, I will do a FULL CLUSTER RESTART. i.e., shutdown all kafka & zookeeper services at once and then start all the services in 5 nodes.

I want to know
1. Does the FULL CLUSTER RESTART distribute the topic partitions evenly across 5 nodes?
Or need to run the partition reassignment tool kafka-reassign-partitions.sh ?
2. Doesn't Kafka have internal mechanism to automatically reassign the partitions unless running kafka-reassign-partitions.sh

Please clarify. Thank you.

1 REPLY 1

Re: Apache Kafka partition reassignment reg.

Contributor

Hello, 

 

Regarding to your questions:

 

1. Does the FULL CLUSTER RESTART distribute the topic partitions evenly across 5 nodes?
All the topics already created will remain the same.
Or need to run the partition reassignment tool kafka-reassign-partitions.sh ?
Yes, this is the available approach.

2. Doesn't Kafka have an internal mechanism to automatically reassign the partitions unless running kafka-reassign-partitions.sh
Unfortunately, this task has to be done manually by using the reassign partition tool.

 

You can check how this tool works by looking at the video article below:

https://www.youtube.com/watch?v=TjKtCKsUjbQ

 

Hope this helps.

Regards,

Manuel.