Hi @balalaika What's your NiFi version? this is a known issue resolved in NiFi 1.2 https://issues.apache.org/jira/browse/NIFI-3213 Even after this, there are several corner situations that the initial design of List* processor try to avoid. For instance, a file being written when List is fired should not be listed. Also, files created just after microsecond after List can be missed if the source system support timestamp with seconds granularity. So it's a tradeoff between missing some files, or delaying ingestion. List processor keeps only the timestamp of the last file ingested and not the list of files (for scalability reasons). To deal with these situations, the last few files are not listed and kept for the next time. The design has been improved in NiFi 1.4, check it out : https://issues.apache.org/jira/browse/NIFI-4069 and https://issues.apache.org/jira/browse/NIFI-3332 Try to investigate on this information and see from where your problems come from. What you can do also is use a cron to run at 3:00 and 3:05 to get files missed the first time (assuming your data comes every 24hours). If you have seconds precisions in your listing then try the new processors in NiFi 1.4 which add a property "Target System Timestamp Precision". Keep in mind that another corner case is not dealt with today which is writing a file at Time T, whith timestamp T2 knowing that T2 < T https://issues.apache.org/jira/browse/NIFI-2383
... View more
