Created on 01-20-2022 03:39 AM - edited 01-20-2022 04:35 AM
how to use ListFTP and FetchFTP processors get all files two hours before the current time in FTP server.
There will be more and more files in the FTP server named by date
These filename in the FTP contains a date as follows:
xxxx-xxxx-xxxx-xxxx-20211231143000-xxxx-xxxx.csv
xxxx-xxxx-xxxx-xxxx-20211231153000-xxxx-xxxx.csv
......
Created on 01-21-2022 09:46 AM - edited 01-21-2022 09:50 AM
Without knowing how the FTP directory is populated, you might have an issue with the ListFTP state and might consider changing its its Listing Strategy,
However assuming that is no issue and you want to only perform an action based on any file with a naming structure like this: xxxx-xxxx-xxxx-xxxx-20220121110000-xxxx-xxxx.csv and only if it is older than two hours from "now"
I would run the output of your ListFTP to an UpdateAttribute and add these 2 properties:
property name
fileTime
value
${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}
property name
timeNow
value
${now():toNumber():minus(7200000)}
Then route that to a RouteOnAttribute and add a new property:
property name value
2 hours old ${fileTime:le(${timeNow})}
Then you can drag that connection to follow on processing and the unmatched connection to other processing or terminate it.
Explanation:
filelName
${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}
This grabs the time out of your filename and converts it to a Date object so it can be converted to its epoch representation
timeNow
${now():toNumber():minus(7200000)}
Sets a value 2 hours ago from current time
${fileTime:le(${timeNow})}
If attribute fileName is less than or equal to timeNow it means that it is 2 hours old.
Created 01-20-2022 10:58 AM
Have you considered looking at the ListFtp property:
Minimum File Age and set it to 2 hours
Created on 01-20-2022 04:31 PM - edited 01-20-2022 05:08 PM
I know the ListSftp processor have 'Minimum File Age' property, but the ListFtp have not 'Minimum File Age' property, do you know how to add 'Minimum File Age' property in ListFtp properssor. thanks!
when i add 'Minimum File Age' in ListFTP processor, it not work, like this:
it displays 'Minimum File Age' is not a supported property or has no Validator associated with it
Created on 01-21-2022 09:46 AM - edited 01-21-2022 09:50 AM
Without knowing how the FTP directory is populated, you might have an issue with the ListFTP state and might consider changing its its Listing Strategy,
However assuming that is no issue and you want to only perform an action based on any file with a naming structure like this: xxxx-xxxx-xxxx-xxxx-20220121110000-xxxx-xxxx.csv and only if it is older than two hours from "now"
I would run the output of your ListFTP to an UpdateAttribute and add these 2 properties:
property name
fileTime
value
${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}
property name
timeNow
value
${now():toNumber():minus(7200000)}
Then route that to a RouteOnAttribute and add a new property:
property name value
2 hours old ${fileTime:le(${timeNow})}
Then you can drag that connection to follow on processing and the unmatched connection to other processing or terminate it.
Explanation:
filelName
${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()}
This grabs the time out of your filename and converts it to a Date object so it can be converted to its epoch representation
timeNow
${now():toNumber():minus(7200000)}
Sets a value 2 hours ago from current time
${fileTime:le(${timeNow})}
If attribute fileName is less than or equal to timeNow it means that it is 2 hours old.