Created 06-05-2018 12:02 PM
I am facing a very similar issue as in the issue https://community.hortonworks.com/questions/141403/in-nifi-tailing-multiple-directories-with-the-sam....
I tried the solution in the above question. The only difference is that there is no 'Rolling Strategy' property available as I'm using nifi 1.6.0 and Rolling Strategy seems to be removed in this commit. Given below is my configuration.
Tailing mode
Multiple files
File(s) to Tail
test[1-3]/flowlog.csv
Rolling Filename Pattern No value set Base directory
C:/cw-data
Initial Start Position
Beginning of File
State Location
Local
Recursive lookup
true
Lookup frequency
5 minutes
Maximum age
24 hours
I'm changing the file C:/cw-data/test1/flowlog.csv, but it is not working.
I'm new in nifi. What am I doing wrong?
Created on 06-05-2018 11:23 PM - edited 08-17-2019 09:20 PM
As you are using Windows os and the specified path(with forward slashes) will work fine for single file as Tailing Mode.
For Multiple files not able to read the regex that you are specifying in files to tail.
I have recreated your scenario and with the below configs i'm able to tail multiple files
Change the configs in your TailFile processor as keep two back slashes(\) on File(s) to Tail and Base directory property values
Tailing mode
Multiple files
File(s) to Tail
test[1-3]\flowlog.csv
Rolling Filename Pattern
flowlog.csv
Base directory
C:\cw-data
Initial Start Position
Beginning of File
State Location
Local
Recursive lookup
true
Lookup frequency
10 minutes
Maximum age
24 hours
If you are using linux the directory path will have forward slashes(/) will work without any issues but in windows directory path will have back slashes we need to keep two back slashes(\) to get correct directory path.
Created on 06-05-2018 11:23 PM - edited 08-17-2019 09:20 PM
As you are using Windows os and the specified path(with forward slashes) will work fine for single file as Tailing Mode.
For Multiple files not able to read the regex that you are specifying in files to tail.
I have recreated your scenario and with the below configs i'm able to tail multiple files
Change the configs in your TailFile processor as keep two back slashes(\) on File(s) to Tail and Base directory property values
Tailing mode
Multiple files
File(s) to Tail
test[1-3]\flowlog.csv
Rolling Filename Pattern
flowlog.csv
Base directory
C:\cw-data
Initial Start Position
Beginning of File
State Location
Local
Recursive lookup
true
Lookup frequency
10 minutes
Maximum age
24 hours
If you are using linux the directory path will have forward slashes(/) will work without any issues but in windows directory path will have back slashes we need to keep two back slashes(\) to get correct directory path.
Created 06-06-2018 08:09 AM
Thanks for your help. That was the problem.