Support Questions
Find answers, ask questions, and share your expertise

How to count files lines using NiFi

Explorer
Hi Everyone, I collect data files with NiFi using ListSFTP AND FetchSFTP processors. In these data files I have several different lines. I want to know if there is a processor that can allow me to count the number of lines in the files as I collect them to have a total number of lines. Someone can help me.
1 ACCEPTED SOLUTION

Rising Star

 I have used CSV file withCountText and it worked fine for me,  Upon success it will writes an attribute name "text.line.count" with number of lines as value, attaching sample csv file which I used to test and it has 6 lines. 

 

Screenshot 2021-06-01 at 9.10.22 PM.png

Screenshot 2021-06-01 at 9.18.35 PM.png

View solution in original post

6 REPLIES 6

Rising Star

You can use processor name CountText

 

Thank You.

Explorer

Hi @ckumar, Thanks for your help but I had used CountText but the problem is that I have csv files and I only see the number of files instead of counting the number of lines in the files.

Rising Star

 I have used CSV file withCountText and it worked fine for me,  Upon success it will writes an attribute name "text.line.count" with number of lines as value, attaching sample csv file which I used to test and it has 6 lines. 

 

Screenshot 2021-06-01 at 9.10.22 PM.png

Screenshot 2021-06-01 at 9.18.35 PM.png

Super Guru

if it's csv, use QueryRecordProcessor with csv reader

 

then just do SELECT COUNT(*) FROM FLOWFILE

Explorer

Hi @TimothySpann, What is the difference between CountText and QueryRecord. Can you detail the use of QueryRecord? Thanks

Super Guru

CountText will count lines (\r\n).

 

QueryRecord will count # of records, even if it is two records on a line

; ;