Nifi Extraction

[#1]

Hi guys,

i would like to ask, if this is possible in nifi alone without using any script execution.

test.txt <--- so this file of mine need to be extracted. Please see the attached file.

I just need to get the VALUE of line1 to line4 and save it to hbase.

The problem here is it has multiple request in 1 file. I need to get all the line1 to line4 per request.

PS. the count of lines per requests are different. My file is just an example of what the file looks like.

Thank you.

[#2 · Accepted solution]

@regie canada

You can use the ExtractText processor and use regex within it to pull the first 4 lines. Your regex would be:

(.*)\n(.*)\n(.*)\n(.*)

After that you can use the SplitText processor if you want each line to be an individual flowfile or you can use the UpdateAttribute processor to make any kind of transformations on the 4 lines.

[#3]

Hi sir, thanks for the reply. I need all the first 4 lines in every ####################################################################### START of Request ####################################################################### sir. it has a multiple request in every file.

[#4]

@Eyad Garelnabi

[#5]

@regie canada

As Matt suggested below, use the SplitContent processor to split the file into multiple, smaller flow files. The "byte sequence" entry for splitting would be

####################################################################### START of Request #######################################################################

After that, use the ExtractText processor, as described in my response above, to get the first 4 lines of each flow file generated by the SplitContent processor.

[#6]

@regie canada

I agree. My answer was only intended to show how to split you multi-record file in to single records to be processed similar to @Eyad Garelnabi suggested approach.

Matt

[#7]

@Matt Clarke @Eyad Garelnabi

Thank you so much!!

[#8]

@regie canada

Does each request in the file always start with

####################################################################### START of Request #######################################################################

and end with:

####################################################################### END of Request #######################################################################

If so, you could use the SplitContent processor to split your incoming FlowFile in multiple FlowFiles (each with a single request). Then you could parse each of those requests for the lines/values you want.

The SplitContent processor would be configured as follows in this scenario:

Do the 4 lines you want to extract the values from have a specific format?

For example do they actually start with "Line" or is that property name always dynamic in nature?

Thanks,

Matt

[#9]

@Matt Clarke

Thanks sir. Anyway for addition question, is there a processor that will change this Line1 : value Line2 : value Line3 : valu Line4: value

to JSON format?

Thank again.

[#10]

@regie canada

The extractText processor creates FlowFile attributes from the extracted text. NiFi has an AttributesToJSON processor you can use to generate JSON form these created attributes.

For new questions, please open a new question. It makes it easier for community users to search for answers.

Thanks,

Matt