question Re: How to count files lines using NiFi in Support Questions

How to count files lines using NiFi

Justee — Tue, 01 Jun 2021 10:46:22 GMT

Hi Everyone, I collect data files with NiFi using ListSFTP AND FetchSFTP processors. In these data files I have several different lines. I want to know if there is a processor that can allow me to count the number of lines in the files as I collect them to have a total number of lines. Someone can help me.

Re: How to count files lines using NiFi

ckumar — Tue, 01 Jun 2021 14:15:01 GMT

You can use processor name CountText

Thank You.

Re: How to count files lines using NiFi

Justee — Tue, 01 Jun 2021 15:22:22 GMT

Hi @ckumar, Thanks for your help but I had used CountText but the problem is that I have csv files and I only see the number of files instead of counting the number of lines in the files.

Re: How to count files lines using NiFi

ckumar — Tue, 01 Jun 2021 15:49:35 GMT

I have used CSV file withCountText and it worked fine for me, Upon success it will writes an attribute name "text.line.count" with number of lines as value, attaching sample csv file which I used to test and it has 6 lines.

Re: How to count files lines using NiFi

TimothySpann — Tue, 01 Jun 2021 17:07:19 GMT

if it's csv, use QueryRecordProcessor with csv reader

then just do SELECT COUNT(*) FROM FLOWFILE

Re: How to count files lines using NiFi

Justee — Tue, 01 Jun 2021 21:19:47 GMT

Hi @TimothySpann, What is the difference between CountText and QueryRecord. Can you detail the use of QueryRecord? Thanks

Re: How to count files lines using NiFi

TimothySpann — Tue, 01 Jun 2021 21:29:49 GMT

CountText will count lines (\r\n).

QueryRecord will count # of records, even if it is two records on a line