Created 06-13-2017 10:00 PM
Created 06-14-2017 05:54 PM
Yes, new brokers can be added while Kafka is online and partitions can be reassigned to these new brokers. One thing to remember is you are not creating new partitions. If you create new partitions, and assuming you have keyed messages where applications require data to be in order then you will lose the order guarantees because in a keyed message Kafka makes sure that a particular key always lands on a particular partition. When you add new partitions, you break this behavior.
But you can easily add new brokers and assign existing partitions to the new brokers in order to balance the cluster.
Created 06-14-2017 05:24 PM
@Neeraj Sabharwal, @Kuldeep Kulkarni, @Josh Elser - any ideas on this ?
Created 06-14-2017 05:54 PM
Yes, new brokers can be added while Kafka is online and partitions can be reassigned to these new brokers. One thing to remember is you are not creating new partitions. If you create new partitions, and assuming you have keyed messages where applications require data to be in order then you will lose the order guarantees because in a keyed message Kafka makes sure that a particular key always lands on a particular partition. When you add new partitions, you break this behavior.
But you can easily add new brokers and assign existing partitions to the new brokers in order to balance the cluster.
Created 06-15-2017 06:57 PM
@mqureshi , thanks !