Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to get files based on the time stamp in Nifi?

Solved Go to solution
Highlighted

how to get files based on the time stamp in Nifi?

Explorer
  1. Please explain how to get files based on time stamp?
  2. How to list a set of files not as (.xls and xlsx) files ie.other than .xls and .xlsx fiiles
1 ACCEPTED SOLUTION

Accepted Solutions

Re: how to get files based on the time stamp in Nifi?

Hi @Iyappan Gopalakrishnan,

If your files are located on local file system you may use ListFile processor. This processor has properties accepting the use of expression language. Using date functions of expression language [1], you can filter the files you list based on the timestamp (if the filenames contain timestamp).

Regarding your second question, you can use the File filter property in ListFile processor which accepts a regular expression. It allows you to exclude some extensions if you don't want to list .xls and .xlsx files.

[1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates

View solution in original post

6 REPLIES 6

Re: how to get files based on the time stamp in Nifi?

Hi @Iyappan Gopalakrishnan,

If your files are located on local file system you may use ListFile processor. This processor has properties accepting the use of expression language. Using date functions of expression language [1], you can filter the files you list based on the timestamp (if the filenames contain timestamp).

Regarding your second question, you can use the File filter property in ListFile processor which accepts a regular expression. It allows you to exclude some extensions if you don't want to list .xls and .xlsx files.

[1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates

View solution in original post

Highlighted

Re: how to get files based on the time stamp in Nifi?

Explorer

Thanks@ Pierre Villard

I'm using Windows Nifi how to list files using time stamp...

in File filter we the default as [^\.].* . we can change * as which ever file format but suppose i don't want .xls file. how can i apply not in [^\.].xls.

Highlighted

Re: how to get files based on the time stamp in Nifi?

- I don't understand what is the problem with Windows. Do your files contain timestamps in the filename? If yes, you can use regular expression to filter on the filename based on the date.

- If you want all files having an extension but with the extension not being equal to .xls or .xlsx, you can use the following regular expression:

^.*\.(?!xls$|xlsx$)[^.]+$

I would recommend you testing your regular expressions on this website : http://regexr.com/

This site provides a "explain tab" giving information regarding the regular expression and how it works.

Highlighted

Re: how to get files based on the time stamp in Nifi?

Explorer

Hi@Pierre Villard

This (^.*\.(?!xls$|xlsx$)[^.]+$) operators is not working for me..it doesn't list any files. in my directory having two excel files and 1 .txt file and .xml file but it not list any....

Highlighted

Re: how to get files based on the time stamp in Nifi?

This works for me.

4813-screen-shot-2016-06-07-at-25047-pm.png

Highlighted

Re: how to get files based on the time stamp in Nifi?

Explorer

now working...

Thanks

Iyappan

Don't have an account?
Coming from Hortonworks? Activate your account here