Member since
10-22-2018
3
Posts
0
Kudos Received
0
Solutions
09-21-2020
05:53 AM
1 Kudo
@dvmishra It always best to start a new thread/question rather than adding a new question to an existing thread that already has an excepted answer. As far as being able to reset the offset for a specific consumer group from within NiFi itself, this is not something that can be done via the ConsumeKafka processors. The offset is not stored by NiFi. Offsets for each consumer group are stored in Kafka. Would not make make much sense to build such an option in to a NiFi processor if it was possible. Every time the processor executes it would reset in that case which is probably not the desired outcome. There are numerous threads online around reseting the offset in Kafka you may want to explore. Here are a couple: https://gist.github.com/marwei/cd40657c481f94ebe273ecc16601674b https://gist.github.com/mduhan/0e0a4b08694f50d8a646d2adf02542fc If you can figure out how to accomplish reset via a custom script of external command, NiFi does offer several script execution and command line execution processors. You may be able to use these processors to execute your script to rest the offset in Kafka. Aside from above, you can change the "group id" (new consumer group) and change the "offset reset" to "earliest". Then restart processor to start consuming topic form beginning again as a different consumer group. Hope this helps, Matt
... View more