Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NIFI - Validate file > 10 rows

Highlighted

NIFI - Validate file > 10 rows

New Contributor

Hello, I need validate the files in my flow, only are valid the files that contains > 10 rows, it´s possible this validation whith apache NIFI?. The files are .txt and .data.

Thanks.

3 REPLIES 3

Re: NIFI - Validate file > 10 rows

Super Guru
@Pepelu Rico

Use QueryRecord processor and define record reader/writer controller services

add new property in the processor as

select id,name from (select count(*) over() cnt,ff.* from FLOWFILE ff)c where cnt > 10

We are using subquery count(*) over() window function on flowfile content to get count of rows from the flowfile then in outer query we are filtering out cnt column and checking the cnt is greater than 10 or not.

I have attached my flow xml here use that as a reference and prepare your sql query as per your requirements.

check-number-of-rows-in-the-flowfile.xml

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

Re: NIFI - Validate file > 10 rows

Expert Contributor

@Pepelu Rico

if for some reason you don't like records (see answer from @Shu, here is easier solution (having fun with regex).

Note, you'll need to set a buffer appropriately, and this will work good if your files are really around 10 lines and lines aren't huge.

92572-flow.png

92573-rounteoncontentprops.png

Re: NIFI - Validate file > 10 rows

Expert Contributor

sorry, missed that you need "greater than 10", so change regex to:

(?s)([^\n]*(\n)){9,}[^\n]*
Note, it is "9" for ten records, because last line won't have \n (EOL)