question Re: Nifi Extraction in Support Questions

Nifi Extraction

regie_canada — Fri, 19 May 2017 13:27:33 GMT

Hi guys,

i would like to ask, if this is possible in nifi alone without using any script execution.

test.txt <--- so this file of mine need to be extracted. Please see the attached file.

I just need to get the VALUE of line1 to line4 and save it to hbase.

The problem here is it has multiple request in 1 file. I need to get all the line1 to line4 per request.

PS. the count of lines per requests are different. My file is just an example of what the file looks like.

Thank you.

Re: Nifi Extraction

egarelnabi — Sun, 18 Aug 2019 09:24:13 GMT

@regie canada

You can use the ExtractText processor and use regex within it to pull the first 4 lines. Your regex would be:

(.*)\n(.*)\n(.*)\n(.*)

After that you can use the SplitText processor if you want each line to be an individual flowfile or you can use the UpdateAttribute processor to make any kind of transformations on the 4 lines.

Re: Nifi Extraction

regie_canada — Tue, 23 May 2017 10:20:35 GMT

Hi sir, thanks for the reply. I need all the first 4 lines in every ####################################################################### START of Request ####################################################################### sir. it has a multiple request in every file.

Re: Nifi Extraction

regie_canada — Tue, 23 May 2017 14:46:12 GMT

@Eyad Garelnabi

Re: Nifi Extraction

MattWho — Sun, 18 Aug 2019 09:24:05 GMT

@regie canada

Does each request in the file always start with

####################################################################### START of Request #######################################################################

and end with:

####################################################################### END of Request #######################################################################

If so, you could use the SplitContent processor to split your incoming FlowFile in multiple FlowFiles (each with a single request). Then you could parse each of those requests for the lines/values you want.

The SplitContent processor would be configured as follows in this scenario:

Do the 4 lines you want to extract the values from have a specific format?

For example do they actually start with "Line" or is that property name always dynamic in nature?

Thanks,

Matt

Re: Nifi Extraction

egarelnabi — Wed, 24 May 2017 00:20:10 GMT

@regie canada

As Matt suggested below, use the SplitContent processor to split the file into multiple, smaller flow files. The "byte sequence" entry for splitting would be

####################################################################### START of Request #######################################################################

After that, use the ExtractText processor, as described in my response above, to get the first 4 lines of each flow file generated by the SplitContent processor.

Re: Nifi Extraction

MattWho — Wed, 24 May 2017 00:25:00 GMT

@regie canada

I agree. My answer was only intended to show how to split you multi-record file in to single records to be processed similar to @Eyad Garelnabi suggested approach.

Matt

Re: Nifi Extraction

regie_canada — Wed, 24 May 2017 12:13:53 GMT

@Matt Clarke @Eyad Garelnabi

Thank you so much!!

Re: Nifi Extraction

regie_canada — Wed, 24 May 2017 14:13:29 GMT

@Matt Clarke

Thanks sir. Anyway for addition question, is there a processor that will change this Line1 : value Line2 : value Line3 : valu Line4: value

to JSON format?

Thank again.

Re: Nifi Extraction

MattWho — Wed, 24 May 2017 19:06:03 GMT

@regie canada

The extractText processor creates FlowFile attributes from the extracted text. NiFi has an AttributesToJSON processor you can use to generate JSON form these created attributes.

For new questions, please open a new question. It makes it easier for community users to search for answers.

Thanks,

Matt