Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi with rolloing file pattern

avatar
Contributor

Hi All,

My rolling log file pattern is something like this

/my/path/directory/my-app-2017-09-06.log
/my/path/directory/my-app-2017-09-07.log
/my/path/directory/my-app-2017-09-08.log

Can you one suggest what can set for property in NIFI for a tailFile processor to read out those. Please note I have old file also and some different file also , but I want to read file wit this specific file name and today onward only , not the old file.

I read the doc available for NiFi in website , but not clear me ...

Can any one please help me out to configure tailFile with this file pattern.

Any help will be highly appreciated actually I stuck on this issue for last 5 days ....

1 ACCEPTED SOLUTION

avatar
Master Guru

Hi @Biswajit Chakrabort,

As you are having rolling logs for daily, i tried to TailFile processor by using File(s) to Tail property as follows

/my/path/directory/my-app-${now():format("yyyy-MM-dd")}.log 
The above expression looks for my-app-2017-09-24.log file in /my/path/directory and tails the file if the file is presented in that directory.

40495-taillogfile.png

View solution in original post

4 REPLIES 4

avatar
Super Guru

@Biswajit Chakraborty

You will use "Rolling filename Pattern" property which in your case can be set to "my-app-*.log. Another thing in your use case you will do is to specify "filesToTail" property. Use expression language to specify your files to tail.

https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates

avatar
Master Guru

Hi @Biswajit Chakrabort,

As you are having rolling logs for daily, i tried to TailFile processor by using File(s) to Tail property as follows

/my/path/directory/my-app-${now():format("yyyy-MM-dd")}.log 
The above expression looks for my-app-2017-09-24.log file in /my/path/directory and tails the file if the file is presented in that directory.

40495-taillogfile.png

avatar
Contributor

Thanks a lot .... it worked exactly as I wanted .... thanks again ... one more thing any link or resource where I can get this kind of information or setup details .....

avatar
Contributor
I believe this will fail if you stop your job today and run it tomorrow.. now will change to other day and you will miss the data...