Support Questions

Find answers, ask questions, and share your expertise

Fetch file last one minute ago from the current time using nifi.

Hi,

Im throwing multiple csv files on my hdfs every minute using logstash.

I need to get the files from the past minute from the current time.

Im using nifi in this process.

For example right now is 11:30 AM, I need to get ONLY all the files that are saved 1 minute ago or 11:29AM.

What is the best approach here using nifi?

Thank you.

1 ACCEPTED SOLUTION

@regie canada

If you're using the GetHDFS processor, you can set the Minimum File Age and Maximum File age properties to

1 minute.

You might have to adjust the properties for a little range, maybe minimum 59 secs and maximum to 61 secs

34431-screen-shot-2017-08-23-at-52358-pm.png

View solution in original post

4 REPLIES 4

@regie canada

If you're using the GetHDFS processor, you can set the Minimum File Age and Maximum File age properties to

1 minute.

You might have to adjust the properties for a little range, maybe minimum 59 secs and maximum to 61 secs

34431-screen-shot-2017-08-23-at-52358-pm.png

@regie canada

Did you try the above solution?

Hi @Wynner, sorry for the late response. I tried it and its working. Thank you so much! But i need to delete the file so it will just fetch 1 time.

@regie canada

The processor can do that, just set the Keep Source File property to false and the file will be deleted.

34613-screen-shot-2017-08-31-at-83255-am.png

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.