Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to set specific time for listsftp to start.

How to set specific time for listsftp to start.

We have many historical number of files on server, but only want to begin listing files from a current date. Right now it appears that we can only clear the state,which starts the listing at the earliest dated file. Nifi version 1.5.0

3 REPLIES 3
Highlighted

Re: How to set specific time for listsftp to start.

Expert Contributor

@Jonathan Bell

ListSFTP doesn't have any built-in way to do this, but what I would suggest to accomplish your task is to use the file.lastModifiedTime attribute that the processor writes into the FlowFiles it creates. That way, you can add a RouteOnAttribute processor immediately downstream of this one that filters out any dates that you are not interested in based on the last time the file was modified. This also allows you to filter on other attributes if you desire, full list of written attributes is contained in the NiFi ListSFTP processor documentation:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

Highlighted

Re: How to set specific time for listsftp to start.

ok.thanks for the clarification. however the modification date seems to be of the flowfile, not the actual file. I tried a file that has a modification date of 6 months ago, but the modified date is current date.

Highlighted

Re: How to set specific time for listsftp to start.

Expert Contributor

That is interesting - in my testing it receives the value of the last modified time in the filesystem you are pulling from. Can you double-check the file you're pulling on your FTP server? Or perhaps try creating a new file and testing?

I tested using this public FTP test site:

ftp://speedtest.tele2.net/

You can see my date matches the date on the FTP site:

78581-screen-shot-2018-06-29-at-162105.png

78582-screen-shot-2018-06-29-at-162125.png

Don't have an account?
Coming from Hortonworks? Activate your account here