NiFi with rolloing file pattern

[#1]

Hi All,

My rolling log file pattern is something like this

/my/path/directory/my-app-2017-09-06.log
/my/path/directory/my-app-2017-09-07.log
/my/path/directory/my-app-2017-09-08.log

Can you one suggest what can set for property in NIFI for a tailFile processor to read out those. Please note I have old file also and some different file also , but I want to read file wit this specific file name and today onward only , not the old file.

I read the doc available for NiFi in website , but not clear me ...

Can any one please help me out to configure tailFile with this file pattern.

Any help will be highly appreciated actually I stuck on this issue for last 5 days ....

[#2]

@Biswajit Chakraborty

You will use "Rolling filename Pattern" property which in your case can be set to "my-app-*.log. Another thing in your use case you will do is to specify "filesToTail" property. Use expression language to specify your files to tail.

https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates

[#3 · Accepted solution]

Hi @Biswajit Chakrabort,

As you are having rolling logs for daily, i tried to TailFile processor by using File(s) to Tail property as follows

/my/path/directory/my-app-${now():format("yyyy-MM-dd")}.log 

The above expression looks for my-app-2017-09-24.log file in /my/path/directory and tails the file if the file is presented in that directory.

[#4]

Thanks a lot .... it worked exactly as I wanted .... thanks again ... one more thing any link or resource where I can get this kind of information or setup details .....

[#5]

I believe this will fail if you stop your job today and run it tomorrow.. now will change to other day and you will miss the data...