question Re: Replace Text search with large file in NIFI in Archives of Support Questions (Read Only)

Replace Text search with large file in NIFI

rustam_kapuria — Sun, 18 Aug 2019 09:52:10 GMT

Hi,

Greetings!

I am using NIFI (1.2.0) & struggling with Replace text processor to bring record in a single line,as for now it split a record in 2 line.
Input record (record-1.png) has Description & filed Value, I want to bring value only in a single line which is replaced by pipe symbol (|). But it split record in 2 lines (split_record.png)
I am using following option for Replace Text ( replace-text.png)
record-1.png
replace-text.png
split-record.png

Re: Replace Text search with large file in NIFI

rustam_kapuria — Sat, 23 Dec 2017 03:34:47 GMT

@Shu: thanks for taking out time. I tired this option, unfortunately it did not work.

Re: Replace Text search with large file in NIFI

rustam_kapuria — Sat, 23 Dec 2017 05:44:16 GMT

@Shu : All attributes are in single line. replacement-value.png

Re: Replace Text search with large file in NIFI

Shu_ashu — Sun, 18 Aug 2019 09:51:57 GMT

@Rustam kapuria

I suspect in your input record having \n for one of the value, so when we do Replace Text processor the resultant content is writing into 2 line.

To resolve this issue you need to use escape json function in replace text processor.

Replacement Value

${ID_NO}|${CHANGE_TITLE:escapeJson()}

Same way as above you need to mention escape json function where you are having \n in json message.

Configs:-

Make sure which json value having \n in it and use escapeJson function for that attribute in Replace Text processor.

Refer below link for more details

https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#escapejson

Here is what i tried:-

Input record:-

{"ID_NO": "8.92" ,"TITLE": "HCC\n"}

Evaluate Json Path:-

Destination

flowfile-attribute

Return Type

auto-detect

Path Not Found Behavior

ignore

Null Value Representation

empty string

CHANGE_TITLE

$.TITLE

ID_NO

$.ID_NO

Output of Eval json path processor:-

Even though i'm having \n after HCC for CHANGE_TITLE,\n is not viewed in attributes tab but when we use CHANGE_TITLE in Replace Text processor it will replace the value with new line.

Replace text processor Configs without escapeJson Function:-

Search Value

(?s)(^.*$)

Replacement Value

${ID_NO}|${CHANGE_TITLE}

Maximum Buffer Size

10 MB

Replacement Strategy

Always Replace

Evaluation Mode

Entire text

Output of replace Text processor without escapejson function:-

As you can see the new line is appeared after replace text processor.

Replace text processor Configs with escapeJson Function:-

Search Value

(?s)(^.*$)

Replacement Value

${ID_NO}|${CHANGE_TITLE:escapeJson()}

Maximum Buffer Size

10 MB

Replacement Strategy

Always Replace

Evaluation Mode

Entire text

Output of replace Text processor with escapejson function:-

as you can see the output is in one line because we have used escapeJson function so the processor escaped the \n and replaced whole content into one line.

Try with escapejson function, if the issue is still not resolved let me know..!!

Re: Replace Text search with large file in NIFI

rustam_kapuria — Tue, 26 Dec 2017 05:32:24 GMT

@Shu : There was a culprit record ending up with new line. Thanks for assisting. escapeJson() is working perfect now. thanks.