question Re: How to fetch header & trailer records from a flowfile in NiFi? in Archives of Support Questions (Read Only)

How to fetch header & trailer records from a flowfile in NiFi?

rohit_ravishank — Sun, 25 Dec 2016 16:33:01 GMT

I have the following records in my file,

0$20011227$20011228$1 1$Suresh 1$Kaamesh 1$Vijay 1$Harish 1$Sabareesh 1$Vaneesh 2$5$1$2$100

The record which starts with '0' is the header record, '1' are the detail records & '2' is the footer record. I need to fetch the header & trailer/footer record to do some validations.

Could you please let me know how this can be achieved in NiFi using regex or some other way?

Re: How to fetch header & trailer records from a flowfile in NiFi?

pacificobuzz — Sun, 25 Dec 2016 22:17:27 GMT

Is each record ended with a newline? Or is it just a long list of records?

Re: How to fetch header & trailer records from a flowfile in NiFi?

Former Member — Sun, 25 Dec 2016 22:30:33 GMT

The simplest way to do this would be an ExtractTest processor. You could configure it like this:

Then you will have the header and footer records in attribute fields called record_header and record_footer, respectively:

Re: How to fetch header & trailer records from a flowfile in NiFi?

rohit_ravishank — Sun, 25 Dec 2016 23:05:34 GMT

Hi David, the lines are terminated by '\n' at the end.

Re: How to fetch header & trailer records from a flowfile in NiFi?

rohit_ravishank — Sun, 25 Dec 2016 23:11:22 GMT

Hi Hellmar,

Thanks for your response, it worked!!

I also tried using the below regex by setting the Multi-line mode to 'True' in the ExtractText processor,

Header - ^[0](.*) Footer - ^[2](.*)

I got the same result. Do you see any issues with this approach?

Re: How to fetch header & trailer records from a flowfile in NiFi?

dpinkston — Sun, 25 Dec 2016 23:21:27 GMT

@Rohit Ravishankar

That should be fine, you could also use splitText processor before the extractText processor. So basically each flowfile would contain a single record.

Re: How to fetch header & trailer records from a flowfile in NiFi?

Former Member — Sun, 25 Dec 2016 23:40:46 GMT

No issues - usually with regexes, there are different ways to reach the same goal.

Re: How to fetch header & trailer records from a flowfile in NiFi?

rohit_ravishank — Sun, 25 Dec 2016 23:54:01 GMT

Thanks Devin!!!

Re: How to fetch header & trailer records from a flowfile in NiFi?

rohit_ravishank — Sun, 25 Dec 2016 23:54:20 GMT

Thanks Hellmar!!!