NiFi - ConsumeKafka(0.10) How to achieve seektobeginning?

[#1]

I understand that for the very first time, we can use the Offset Reset to earliest would do. However, subsequently if for load testing and validations if the offset needs to be reset to the beginning, how can we achieve that through NiFi?

[#2 · Accepted solution]

Hello Kumar,

There's an ongoing effort to provide a feature to reset Kafka offset from NiFi UI. We're trying to provide a way for NiFi user to reset state, that is managed externally, such as Kafka offset.

https://issues.apache.org/jira/browse/NIFI-2078

https://github.com/apache/nifi/pull/563

Specifically, clear offset by submitting new offset as -1.

https://github.com/apache/nifi/pull/563/files#diff-c473fdb0ebb08dd72a64d0e6536e85c0R396

I hope I'll be able to resume working on that JIRA soon, please watch the JIRA issue to be updated progress.

[#3]

Koji Kawamura, Did you get chance to work on the offset reset feature? Having ability to reset offset to a target value is necessary in case the pipeline needs to be reprocessed.

[#4]

Hi @AViradia, No, I haven't. But I just found this Stackoverflow. If you're using ConsumeKafka, update the processor to use new consumer group id would be a simple work-around to reprocess from the beginning. http://stackoverflow.com/questions/37741936/how-to-delete-kafka-consumer-created-via-new-consumer-ap...