In Nifi is there anyway to consume kafka messages from beginning like we do from kafka console consumer or from next offset which last consumed?

[#1]

In Nifi is there anyway to consume kafka messages from beginning like we do from kafka console consumer? Apart from consuming messages when Nifi has started it will receive current messages which are produced after the application has started but if i need to consume messages while my application was offline or from next offset which last consumed? Thank you

[#2 · Accepted solution]

There is a property on the GetKafka processor which species the offset to start reading from. "Auto Offset Reset" should be set to "smallest" to provide the equivalent of --from-beginning in the console consumer.

[#3]

@Simon Elliston BallDoes Nifi store offsets in memory as well. Just wondering how will it make sure not to read same offsets or duplicate messages from kafka? thanks