Support Questions

Find answers, ask questions, and share your expertise

Is there a tool to export offsets?

avatar
Explorer
 
1 ACCEPTED SOLUTION

avatar
Super Guru

@Connor O'Neal

There is no named script to export offsets, but we are able to use the kafka-run-class.sh script to execute the underlying Java class for the tool in the proper environment. Exporting offsets will produce a file that contains each topic partition for the group and it’s offsets in a defined format that the import tool can read. The file that is created will have one topic partition per line, with the following format:

/consumers/GROUPNAME/offsets/topic/TOPICNAME/PARTITIONID-0:OFFSET 

Let's assume export the offsets for the consumer group named “testconsumergroup” to a file named “offsets”:

# kafka-run-class.sh kafka.tools.ExportZkOffsets --zkconnect mykafka.abc.com:2181/kafka-cluster --group testconsumergroup --output-file offsets

# cat offsets

/consumers/testconsumergroup/offsets/my-topic/0:8905 /consumers/testconsumergroup/offsets/my-topic/1:8915 /consumers/testconsumergroup/offsets/my-topic/2:9845 /consumers/testconsumergroup/offsets/my-topic/3:8072 /consumers/testconsumergroup/offsets/my-topic/4:8008 /consumers/testconsumergroup/offsets/my-topic/5:8319 /consumers/testconsumergroup/offsets/my-topic/6:8102 /consumers/testconsumergroup/offsets/my-topic/7:12739

+++

If this helped, please vote/accept response.

View solution in original post

1 REPLY 1

avatar
Super Guru

@Connor O'Neal

There is no named script to export offsets, but we are able to use the kafka-run-class.sh script to execute the underlying Java class for the tool in the proper environment. Exporting offsets will produce a file that contains each topic partition for the group and it’s offsets in a defined format that the import tool can read. The file that is created will have one topic partition per line, with the following format:

/consumers/GROUPNAME/offsets/topic/TOPICNAME/PARTITIONID-0:OFFSET 

Let's assume export the offsets for the consumer group named “testconsumergroup” to a file named “offsets”:

# kafka-run-class.sh kafka.tools.ExportZkOffsets --zkconnect mykafka.abc.com:2181/kafka-cluster --group testconsumergroup --output-file offsets

# cat offsets

/consumers/testconsumergroup/offsets/my-topic/0:8905 /consumers/testconsumergroup/offsets/my-topic/1:8915 /consumers/testconsumergroup/offsets/my-topic/2:9845 /consumers/testconsumergroup/offsets/my-topic/3:8072 /consumers/testconsumergroup/offsets/my-topic/4:8008 /consumers/testconsumergroup/offsets/my-topic/5:8319 /consumers/testconsumergroup/offsets/my-topic/6:8102 /consumers/testconsumergroup/offsets/my-topic/7:12739

+++

If this helped, please vote/accept response.