question File Watcher scenario in HDF in Archives of Support Questions (Read Only)

File Watcher scenario in HDF

csankaraiah — Tue, 21 Apr 2026 12:34:08 GMT

Can we have a file watcher kind of mechanism in Nifi, where the data flow gets triggered when ever a file shows up at source? Is it same as scheduling a getfile processor or run always?

Re: File Watcher scenario in HDF

bbende — Fri, 20 Nov 2015 07:37:24 GMT

A GetFile processor with the default scheduling (0 secs = run as fast as possible) should handle this. A common scenario that comes up is the idea of picking up only new files that have been placed in a directory, but never removing any of them. This will eventually be accomplished through ListFile and FetchFile processors (open pull-request for ListFile right now). ListFile will maintain state of what was seen on previous executions, and provide FetchFile with the new files to retrieve.

Re: File Watcher scenario in HDF

rgelhausen — Fri, 20 Nov 2015 11:02:24 GMT

https://issues.apache.org/jira/browse/NIFI-631 for reference for ListFile and FetchFile

Re: File Watcher scenario in HDF

jstraub — Fri, 20 Nov 2015 13:49:22 GMT

You should definitely talk to @nmaillard he is developing a File-Notification Processor that is capable of doing that. I think it gets triggered when new files show up in HDFS (not sure about changes) and you have access to different file attributes.

Re: File Watcher scenario in HDF

csankaraiah — Fri, 20 Nov 2015 19:58:19 GMT

Thanks @bbende and @Jonas Straub