We have many historical number of files on server, but only want to begin listing files from a current date. Right now it appears that we can only clear the state,which starts the listing at the earliest dated file. Nifi version 1.5.0
ListSFTP doesn't have any built-in way to do this, but what I would suggest to accomplish your task is to use the file.lastModifiedTime attribute that the processor writes into the FlowFiles it creates. That way, you can add a RouteOnAttribute processor immediately downstream of this one that filters out any dates that you are not interested in based on the last time the file was modified. This also allows you to filter on other attributes if you desire, full list of written attributes is contained in the NiFi ListSFTP processor documentation:
ok.thanks for the clarification. however the modification date seems to be of the flowfile, not the actual file. I tried a file that has a modification date of 6 months ago, but the modified date is current date.
That is interesting - in my testing it receives the value of the last modified time in the filesystem you are pulling from. Can you double-check the file you're pulling on your FTP server? Or perhaps try creating a new file and testing?
I tested using this public FTP test site:
You can see my date matches the date on the FTP site: