New Contributor
Posts: 3
Registered: ‎08-05-2015

How KafkaSource can scale on Flume ?

Assuming I have a single Kafka topic with 4 partitions then I can scale by adding up to 4 KafkaSource, each one will read a partition of my single topic.



Now, assuming I have 4 topics, each one with 4 partitions. 

I have created a new KafkaSource able to read from multiple topics by using wildcard :


In this case, If I add new KafkaSource does it scale correctly ? I mean, do the available partition/topic will  be dispatched among my KafkaSource ?

If yes, in that case it means I can add up to 16 KafkaSource ?

New solutions