kafka-consumer-groups.sh vs ConsumerOffsetChecker

[#1]

Hi All - I've Kafka 0.9 (going forward will be migrating to Kafka 0.10) & trying to use the ConsumerOffsetChecker & bin/kafka-consumer-groups.sh to check for offsets.

I'm seeing different behavior in using the above 2 scrripts.

Here is what i did -> a) When i use ConsumerOffsetChecker

$KAFKA_HOME/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group myGroup --topic newBroker1

[2017-06-15 11:50:23,729] WARN WARNING: ConsumerOffsetChecker is deprecated and will be dropped in releases following 0.9.0. Use ConsumerGroupCommand instead. (kafka.tools.ConsumerOffsetChecker$)

Group Topic Pid Offset logSize Lag Owner

myGroup newBroker1 0 224216 507732 283516 none

myGroup newBroker1 1 224888 508978 284090 none

myGroup newBroker1 2 141104 424038 282934 none

myGroup newBroker1 3 11829 295110 283281 none

myGroup newBroker1 4 11580 294510 282930 none

b) When i run the kafka-consumer-groups.sh (since the WARNING message above says - ConsumerOffsetChecker is deprecated & we need to use ConsumerGroupCommand instead)

$KAFKA_HOME/bin/kafka-consumer-groups.sh --describe --bootstrap-server localhost:9092,localhost:9093,localhost:9094,localhost:9095 --new-consumer --group myGroup

Consumer group `myGroup` does not exist or is rebalancing.

OR

$KAFKA_HOME/bin/kafka-consumer-groups.sh --describe --zookeeper localhost:2181 --group myGroup

No topic available for consumer group provided

GROUP, TOPIC, PARTITION, CURRENT OFFSET, LOG END OFFSET, LAG, OWNER

The question - When i use $KAFKA_HOME/bin/kafka-consumer-groups.sh - why is the script not giving me Offset details of the group - myGroup & Topic - newBroker1

What needs to be done for this (using script $KAFKA_HOME/bin/kafka-consumer-groups.sh) ?

[#2]

@mqureshi, @amankumbare - looping you in, any ideas ?

[#3]

@Karan Alang

What you are doing is right. "--new-consumer" should make it work if you are using the new consumer. Can you please try again the following.

kafka-run-class.sh kafka.admin.ConsumerGroupCommand --new-consumer --describe --group myGroup --zookeeper localhost:2181

But if this doesn't help, check your zookeeper.

Run zookeeper cli. do an ls on /consumers/myGroup/offsets/<topicname>

You should get some thing like [0].

Then run a get command on this offset. Something like:

get /consumers/myGroup/offsets/<topicname>/0

Then you'll see the first thing is your offset. If it's there, then paste the result and we'll go from there.