Hi Team, we noticed ListS3 processor is processing same file twice or more, while there was no change in the source file.
As we go though the "Component State" which clearly says, nifi manages the file states with latest time-stamp and list only when a new file has been added or existing file has been modified.
We are seeing ListS3 processor is reprocessing same file from Data Provenance which has the same s3.lastModifed property value, We believe file should not be re-processed unless its changed per "State Management" docs.
Kindly help us on this issue. Let us know for any information needed.
Note: ListS3 Processor has been setup to run only primary node.
Yes a code fix in NIFI is needed. Work around is DetectDuplicate processor.