question Re: Adding New Kafka Brokers: Question in Support Questions

Adding New Kafka Brokers: Question

DataMike — Wed, 19 Oct 2022 16:54:42 GMT

Hi all, this may be an easy question, but just wanted to verify. When adding new brokers to the cluster, after installing the Kafka role/instance on the new hosts, should you start the new broker instances first before you restart the existing brokers? Or does it matter? This is CDP Private Cloud 7.1.7

Thanks

Re: Adding New Kafka Brokers: Question

Bharati — Wed, 19 Oct 2022 17:48:06 GMT

AFAIU It should not matter. You can choose the rolling restart option as well. So CM can decide the sequence of the broker restarts.

Re: Adding New Kafka Brokers: Question

Babasaheb — Wed, 19 Oct 2022 17:52:25 GMT

@DataMike

The broker restarting order will not cause any issues. But after adding new brokers you should rebalance topics/partitions between all available brokers for better performance. Refer the following article for more details:

https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/kafka-managing/topics/kafka-manage-cli-reassign-use.html

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

Re: Adding New Kafka Brokers: Question

DataMike — Wed, 19 Oct 2022 18:01:52 GMT

Thanks. I think I can use Cruise Control to re-distribute the partitions across the new brokers. Haven't tried yet though.

Re: Adding New Kafka Brokers: Question

Babasaheb — Wed, 19 Oct 2022 18:22:53 GMT

@DataMike

Yes, you can use the CC APIs as per your requirements [1] [2]

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/cctrl-managing/topics/cctrl-using-rest-api.html

[2] https://community.cloudera.com/t5/Customer/Frequently-Used-CRUISE-CONTROL-API-and-important-DOCs/ta-p/324729

From the above articles you can use the following API to rebalance topics/partitions:

curl -k --negotiate -u: -X POST "https://<CC FQDN>:8899/kafkacruisecontrol/rebalance?dryrun=false&rebalance_disk=true"

To avoid high CPU, memory and disk read write you can initiate rebalance process in batches. It will automatically create a batch and rebalance topics.

curl -X POST "http://$HOSTNAME:8899/kafkacruisecontrol/rebalance?dryrun=true&concurrent_partition_movements_per_broker=10&concurrent_leader_movements=500"

If it will help you then please click on ”Accept as Solution" below this post.

Thank you.