Improve Tailfile Processor for use patterns in the "File to Tail" property ?

[#1]

Hi,

I've got more than one hundred files to tail and I would like to use patterns in the File to Tail property.

For example : /donapp/*/p00/wi*/log/*.?.*log.2016-06-22

Do you think it's possible to improve this processor ? Or is there another way to do it ?

Thanks

[#2 · Accepted solution]

Hi @Thierry Vernhet,

NiFi is an Apache software, for improvement wish I'd recommend you to raise a JIRA on https://issues.apache.org/jira/browse/NIFI or to subscribe to dev mailing list (https://nifi.apache.org/mailing_lists.html) and start a discussion on this list about this subject.

In any case, IMO it seems to be a valid point but it needs to go through Apache process.

Hope this helps.

[#3]

Hi @Pierre Villard

Thanks for your answer.

[#4]

@Thierry Vernhet

Is the intent that it could use patterns to find a file to tail or is it that it will find the file(s) to tail? If it is meant to be about finding a single file then this seems doable. If the idea is to match on multiple files at once can you share how you expect it to behave?

[#5]

Hi @jwit

The idea is to match on multiple files. Instead of create several processors with one file to tail for each one, I'd like to create only one processor with a "pattern" to tail which adresses several files.

Obviously, we except for each flow ingested the right properties "path, filename ... " of the original file tailed in this flow. So the properties values may change with each flow.

Let me know if it's clear to you.

Thanks