Hello All,
We're using Apache NiFi 1.0.1, I know we're way behind in upgrading.
Our use case is to get files from a local NiFi server mount and write to HDFS; we're using ListFile and FetchFile to achieve this. Some files are huge, so the concern is that NiFi might start to fetch files before they're completely written to the mount, which would cause partial file loads in HDFS. So the solution proposed is, the source system would send us an indicator file (located on a different directory) with a specific name; once we get that file, then we should start fetching the files with FetchFile processor.
So, the question is, how do we build the NiFi dataflow in such a way that FetchFile will only start after the indicator file is received.
Do you have any suggestions on how to achieve this.
Thanks in advance.