Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

TailFile with multiple files

Highlighted

TailFile with multiple files

New Contributor

In the following path, i have the following logs:

When 10MB are reached for metricbeat file, it will go to write on metricbeat.1 and when this file size exceeds also, it will write to metricbeat.2 etc

[root@ metricbeat]# ls -lorth
total 4.1M
-rwxrwxrwx 1 nifi   10 Aug 17 11:17 metricbeat.2
-rwxrwxrwx 1 nifi   10 Aug 17 11:17 metricbeat.1
-rwxrwxrwx 1 nifi 4.1M Aug 17 11:47 metricbeat

In NiFi (no cluster) I want to tail all the files that are stored on path

/logs/metricbeat/

I am using TailFile Processor with the following Configuration:

TailFile NiFi Processor Configuration

85672-nifi.png

But, the main problem is that i am getting the following error:

'File to Tail' is invalid because There is no file to tail. Files must exist when starting this processor.

85673-nifi2.png

If i select the "Single Line" it gets the file "metricbeat"

Could you please tell me what i am doing wrong? Or how can i read all the "metricbeat" files from that path?

1 REPLY 1

Re: TailFile with multiple files

Expert Contributor

@Octavian Duta

In TailFile set the following Attributes:

File(s) to Tail: /logs/metricbeat

Rolling FileName Pattern: metricbeat.*

Base Directory: No value Set (leave it EMPTY)

If this answer is helpful please choose Accept to mark your question as answered.