question Re: NiFi with rolloing file pattern in Support Questions

NiFi with rolloing file pattern

Biswajit16 — Sun, 24 Sep 2017 00:15:46 GMT

Hi All,

My rolling log file pattern is something like this

/my/path/directory/my-app-2017-09-06.log /my/path/directory/my-app-2017-09-07.log /my/path/directory/my-app-2017-09-08.log

Can you one suggest what can set for property in NIFI for a tailFile processor to read out those. Please note I have old file also and some different file also , but I want to read file wit this specific file name and today onward only , not the old file.

I read the doc available for NiFi in website , but not clear me ...

Can any one please help me out to configure tailFile with this file pattern.

Any help will be highly appreciated actually I stuck on this issue for last 5 days ....

Re: NiFi with rolloing file pattern

mqureshi — Sun, 24 Sep 2017 01:41:51 GMT

@Biswajit Chakraborty

You will use "Rolling filename Pattern" property which in your case can be set to "my-app-*.log. Another thing in your use case you will do is to specify "filesToTail" property. Use expression language to specify your files to tail.

https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates

Re: NiFi with rolloing file pattern

Shu_ashu — Sun, 18 Aug 2019 05:52:47 GMT

Hi @Biswajit Chakrabort,

As you are having rolling logs for daily, i tried to TailFile processor by using File(s) to Tail property as follows

/my/path/directory/my-app-${now():format("yyyy-MM-dd")}.log

The above expression looks for my-app-2017-09-24.log file in /my/path/directory and tails the file if the file is presented in that directory.

Re: NiFi with rolloing file pattern

Biswajit16 — Mon, 25 Sep 2017 15:59:31 GMT

Thanks a lot .... it worked exactly as I wanted .... thanks again ... one more thing any link or resource where I can get this kind of information or setup details .....

Re: NiFi with rolloing file pattern

reyaan26 — Wed, 16 Sep 2020 22:27:00 GMT

I believe this will fail if you stop your job today and run it tomorrow.. now will change to other day and you will miss the data...