Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to count files lines using NiFi

avatar
Explorer
Hi Everyone, I collect data files with NiFi using ListSFTP AND FetchSFTP processors. In these data files I have several different lines. I want to know if there is a processor that can allow me to count the number of lines in the files as I collect them to have a total number of lines. Someone can help me.
1 ACCEPTED SOLUTION

avatar
Master Collaborator

 I have used CSV file withCountText and it worked fine for me,  Upon success it will writes an attribute name "text.line.count" with number of lines as value, attaching sample csv file which I used to test and it has 6 lines. 

 

Screenshot 2021-06-01 at 9.10.22 PM.png

Screenshot 2021-06-01 at 9.18.35 PM.png

View solution in original post

6 REPLIES 6

avatar
Master Collaborator

You can use processor name CountText

 

Thank You.

avatar
Explorer

Hi @ckumar, Thanks for your help but I had used CountText but the problem is that I have csv files and I only see the number of files instead of counting the number of lines in the files.

avatar
Master Collaborator

 I have used CSV file withCountText and it worked fine for me,  Upon success it will writes an attribute name "text.line.count" with number of lines as value, attaching sample csv file which I used to test and it has 6 lines. 

 

Screenshot 2021-06-01 at 9.10.22 PM.png

Screenshot 2021-06-01 at 9.18.35 PM.png

avatar
Master Guru

if it's csv, use QueryRecordProcessor with csv reader

 

then just do SELECT COUNT(*) FROM FLOWFILE

avatar
Explorer

Hi @TimothySpann, What is the difference between CountText and QueryRecord. Can you detail the use of QueryRecord? Thanks

avatar
Master Guru

CountText will count lines (\r\n).

 

QueryRecord will count # of records, even if it is two records on a line