Support Questions
Find answers, ask questions, and share your expertise

Can I use same consumer group Id for two different topic? Also, how many consumer will be created in one group id?

Can I use same consumer group Id for two different topic? Also, how many consumer will be created in one group id?

There are two different topic and each topic have 30 partitions and two different producer is producing data in these topics. Also, there are two different kafka consumer but with same group Id. Will it create any problem? Also, how many consumer instance will be created under one group id? Please help me to understand this.

3 REPLIES 3

Re: Can I use same consumer group Id for two different topic? Also, how many consumer will be created in one group id?

Super Collaborator

Hi Shailendra,

Having 2 Kafka consumers with the same group ID will be just fine. This is by design actually. Kafka internals will try to load balance the topic consumption between any consumers registering on the group ID.

The number of consumers per group ID is not bound to anything, you can have as many as you want. Usually the consuming application (like Storm) sets/decides this. For optimal performance you would typically have as many consumers registering for a group ID as there are topic partitions. That way, a consumer would not have to switch between multiple assigned partitions, but would just stick to consuming from the same partition during its life span, which is most efficient.

Re: Can I use same consumer group Id for two different topic? Also, how many consumer will be created in one group id?

Super Collaborator

@Shailendra Lohia

And yes, you could use the same group ID for many different topics, if you want. For Kafka that would be transparent, as it would just be totally different 'keys' (topic:consumer_group_id) to Kafka

Re: Can I use same consumer group Id for two different topic? Also, how many consumer will be created in one group id?

Super Collaborator

@Shailendra Lohia Please mark the question as answered if sufficiently answered, thank you