Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to fetch header & trailer records from a flowfile in NiFi?

avatar
Contributor

I have the following records in my file,

0$20011227$20011228$1 1$Suresh 1$Kaamesh 1$Vijay 1$Harish 1$Sabareesh 1$Vaneesh 2$5$1$2$100

The record which starts with '0' is the header record, '1' are the detail records & '2' is the footer record. I need to fetch the header & trailer/footer record to do some validations.

Could you please let me know how this can be achieved in NiFi using regex or some other way?

1 ACCEPTED SOLUTION

avatar

The simplest way to do this would be an ExtractTest processor. You could configure it like this:

Then you will have the header and footer records in attribute fields called record_header and record_footer, respectively:

View solution in original post

8 REPLIES 8

avatar
New Contributor

Is each record ended with a newline? Or is it just a long list of records?

avatar
Contributor

Hi David, the lines are terminated by '\n' at the end.

avatar

The simplest way to do this would be an ExtractTest processor. You could configure it like this:

Then you will have the header and footer records in attribute fields called record_header and record_footer, respectively:

avatar
Contributor

Hi Hellmar,

Thanks for your response, it worked!!

I also tried using the below regex by setting the Multi-line mode to 'True' in the ExtractText processor,

Header - ^[0](.*) Footer - ^[2](.*)

I got the same result. Do you see any issues with this approach?

avatar
Rising Star

@Rohit Ravishankar

That should be fine, you could also use splitText processor before the extractText processor. So basically each flowfile would contain a single record.

avatar
Contributor

Thanks Devin!!!

avatar

No issues - usually with regexes, there are different ways to reach the same goal.

avatar
Contributor

Thanks Hellmar!!!