Fetch file last one minute ago from the current time using nifi.

[#1]

Hi,

Im throwing multiple csv files on my hdfs every minute using logstash.

I need to get the files from the past minute from the current time.

Im using nifi in this process.

For example right now is 11:30 AM, I need to get ONLY all the files that are saved 1 minute ago or 11:29AM.

What is the best approach here using nifi?

Thank you.

[#2 · Accepted solution]

@regie canada

If you're using the GetHDFS processor, you can set the Minimum File Age and Maximum File age properties to

1 minute.

You might have to adjust the properties for a little range, maybe minimum 59 secs and maximum to 61 secs

[#3]

@regie canada

Did you try the above solution?

[#4]

Hi @Wynner, sorry for the late response. I tried it and its working. Thank you so much! But i need to delete the file so it will just fetch 1 time.

[#5]

@regie canada

The processor can do that, just set the Keep Source File property to false and the file will be deleted.