Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Nifi track files already processed?

avatar
New Member

I have an NFS read only Share that contains thousands of unique files (all with unique names) which I want to process with Nifi. New files are constantly added to this share by another process. Is there any way to keep track of which files Nifi already processes in a prior run, so I don't process them again? I cannot make any changes to files on files on this NFS share (cannot delete or rename them).

1 ACCEPTED SOLUTION

avatar
Master Guru

The ListFile processor keeps track of files it has already seen and only picks up files where the modified date is newer than the last time the processor ran. ListFile produces a flow file for each path to fetch and is used with FetchFile to actually retrieve the file.

View solution in original post

2 REPLIES 2

avatar
Master Guru

The ListFile processor keeps track of files it has already seen and only picks up files where the modified date is newer than the last time the processor ran. ListFile produces a flow file for each path to fetch and is used with FetchFile to actually retrieve the file.

avatar
New Member

Thanks Bryan, thats just what I was looking for!