Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to get files two hours ago in FTP file system,thanks

avatar
Explorer

how to use ListFTP and FetchFTP processors get all files two hours before the current time in FTP server.

There will be more and more files in the FTP server named by date

These filename in the FTP contains a date as follows: 

xxxx-xxxx-xxxx-xxxx-20211231143000-xxxx-xxxx.csv

xxxx-xxxx-xxxx-xxxx-20211231153000-xxxx-xxxx.csv

......

 

 

 

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Without knowing how the FTP directory is populated, you might have an issue with the ListFTP state and might consider changing its its Listing Strategy,

However assuming that is no issue and you want to only perform an action based on any file with a naming structure like this: xxxx-xxxx-xxxx-xxxx-20220121110000-xxxx-xxxx.csv and only if it is older than two hours from "now"

I would run the output of your ListFTP to an UpdateAttribute and add these 2 properties:

 

property name       

fileTime         

value

${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}

property name

timeNow                     

value

${now():toNumber():minus(7200000)}

 

Then route that to a RouteOnAttribute and add a new property:

 

property name              value

2 hours old                ${fileTime:le(${timeNow})}

 

Then you can drag that connection to follow on processing and the unmatched connection to other processing or terminate it.

 

Explanation:

filelName

${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}

This grabs the time out of your filename and converts it to a Date object so it can be converted to its epoch representation

 

timeNow

${now():toNumber():minus(7200000)}

Sets a value 2 hours ago from current time

 

 

${fileTime:le(${timeNow})}

If attribute fileName is less than or equal to timeNow it means that it is 2 hours old.

 

 

 

View solution in original post

3 REPLIES 3

avatar
Expert Contributor

Have you considered looking at the ListFtp property:

 

Minimum File Age and set it to 2 hours

avatar
Explorer

I know the ListSftp processor have 'Minimum File Age' property, but the ListFtp have not  'Minimum File Age' property, do you know how to add 'Minimum File Age' property  in ListFtp properssor. thanks!

when i add 'Minimum File Age' in ListFTP processor,  it not work, like this:

zhangliang_0-1642727244943.png

it displays 'Minimum File Age' is not a supported property or has no Validator associated with it 

avatar
Expert Contributor

Without knowing how the FTP directory is populated, you might have an issue with the ListFTP state and might consider changing its its Listing Strategy,

However assuming that is no issue and you want to only perform an action based on any file with a naming structure like this: xxxx-xxxx-xxxx-xxxx-20220121110000-xxxx-xxxx.csv and only if it is older than two hours from "now"

I would run the output of your ListFTP to an UpdateAttribute and add these 2 properties:

 

property name       

fileTime         

value

${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}

property name

timeNow                     

value

${now():toNumber():minus(7200000)}

 

Then route that to a RouteOnAttribute and add a new property:

 

property name              value

2 hours old                ${fileTime:le(${timeNow})}

 

Then you can drag that connection to follow on processing and the unmatched connection to other processing or terminate it.

 

Explanation:

filelName

${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}

This grabs the time out of your filename and converts it to a Date object so it can be converted to its epoch representation

 

timeNow

${now():toNumber():minus(7200000)}

Sets a value 2 hours ago from current time

 

 

${fileTime:le(${timeNow})}

If attribute fileName is less than or equal to timeNow it means that it is 2 hours old.