question Re: Nifi track files already processed? in Archives of Support Questions (Read Only)

Nifi track files already processed?

edmund_prout — Mon, 15 Aug 2016 21:12:04 GMT

I have an NFS read only Share that contains thousands of unique files (all with unique names) which I want to process with Nifi. New files are constantly added to this share by another process. Is there any way to keep track of which files Nifi already processes in a prior run, so I don't process them again? I cannot make any changes to files on files on this NFS share (cannot delete or rename them).

Re: Nifi track files already processed?

bbende — Mon, 15 Aug 2016 21:15:25 GMT

The ListFile processor keeps track of files it has already seen and only picks up files where the modified date is newer than the last time the processor ran. ListFile produces a flow file for each path to fetch and is used with FetchFile to actually retrieve the file.

Re: Nifi track files already processed?

edmund_prout — Mon, 15 Aug 2016 21:19:56 GMT

Thanks Bryan, thats just what I was looking for!