Agree with @MattWho . You can use the nifi expression language to choose the certain files from your source. Based on the date or file name you can filter out the files using FetchSFTP processor. i can see in the latest version of nifi expression language is false in ListSFTP processor. to handle your case you have to do it in two steps ListSFTP ==> FetchSFTP (If you want to put dynamic date filters in your source directories).   e.g. you can mention the below in Remote file property of FetchSFTP 0000000001_${now():toNumber():minus(86400000):format('yyyyMMdd')}235959_filename this will gives us output as 0000000001_20200112235959_filename.   Regards Nitin     ... View more

Hi, I have got some log messages like this when I cannot get a downloaded files (there's no errors flow: comms failed, not found, permission denied) . I wonder whether this case related to the setting of "Data timeout" and "Connection timeout". If you think so, please suggest, how can I deal with this problem? What if I put 0 sec to each parameter, does this mean no limit timeout? Thank you 2018-10-18 08:59:50,661 INFO [Provenance Maintenance Thread-2] o.a.n.p.expiration.FileRemovalAction Removed expired Provenance Event file /nifi-repository-provenance/64054866.prov.gz ... View more