Support Questions
Find answers, ask questions, and share your expertise

NiFi with rolloing file pattern

Solved Go to solution

NiFi with rolloing file pattern

Explorer

Hi All,

My rolling log file pattern is something like this

/my/path/directory/my-app-2017-09-06.log
/my/path/directory/my-app-2017-09-07.log
/my/path/directory/my-app-2017-09-08.log

Can you one suggest what can set for property in NIFI for a tailFile processor to read out those. Please note I have old file also and some different file also , but I want to read file wit this specific file name and today onward only , not the old file.

I read the doc available for NiFi in website , but not clear me ...

Can any one please help me out to configure tailFile with this file pattern.

Any help will be highly appreciated actually I stuck on this issue for last 5 days ....

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NiFi with rolloing file pattern

Super Guru

Hi @Biswajit Chakrabort,

As you are having rolling logs for daily, i tried to TailFile processor by using File(s) to Tail property as follows

/my/path/directory/my-app-${now():format("yyyy-MM-dd")}.log 
The above expression looks for my-app-2017-09-24.log file in /my/path/directory and tails the file if the file is presented in that directory.

40495-taillogfile.png

View solution in original post

4 REPLIES 4
Highlighted

Re: NiFi with rolloing file pattern

Super Guru

@Biswajit Chakraborty

You will use "Rolling filename Pattern" property which in your case can be set to "my-app-*.log. Another thing in your use case you will do is to specify "filesToTail" property. Use expression language to specify your files to tail.

https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates

Highlighted

Re: NiFi with rolloing file pattern

Super Guru

Hi @Biswajit Chakrabort,

As you are having rolling logs for daily, i tried to TailFile processor by using File(s) to Tail property as follows

/my/path/directory/my-app-${now():format("yyyy-MM-dd")}.log 
The above expression looks for my-app-2017-09-24.log file in /my/path/directory and tails the file if the file is presented in that directory.

40495-taillogfile.png

View solution in original post

Highlighted

Re: NiFi with rolloing file pattern

Explorer

Thanks a lot .... it worked exactly as I wanted .... thanks again ... one more thing any link or resource where I can get this kind of information or setup details .....

Highlighted

Re: NiFi with rolloing file pattern

Contributor
I believe this will fail if you stop your job today and run it tomorrow.. now will change to other day and you will miss the data...