question Re: URGENT query regarding ReplaceText error and processing in Archives of Support Questions (Read Only)

URGENT query regarding ReplaceText error and processing

sahaavish — Sun, 18 Aug 2019 06:08:38 GMT

I am using ReplaceText to replace JSON and avoid unnecessary escaping of JSON arrays.

However, it looks like there is some inherent problem in either the way I am using ReplaceText or with ReplaceText itself. I always end up having below error while processing of JSON data which I first Split into single items, usually at the end of the processing when fewer flow files are left in the queue and are always penalized for some reason. Please find below the error screenshot -

Re: URGENT query regarding ReplaceText error and processing

Wynner — Sun, 18 Aug 2019 06:08:30 GMT

@Avish Saha

If you right click on the incoming connection a pop-up menu will appear:

Select List queue, then you can look at the data the ReplaceText is trying to process. It looks like the end of the file being split may have incomplete lines?

Re: URGENT query regarding ReplaceText error and processing

sahaavish — Mon, 24 Apr 2017 20:45:05 GMT

Thank you for the response, it helped. Though one thing I dont get is why ReplaceText waits forever as soon as an error occurs and it penalizes a flow file. Any clues or if anyone has seen this behavior ?

Please advice. Thank you.

Re: URGENT query regarding ReplaceText error and processing

MattWho — Mon, 24 Apr 2017 21:15:59 GMT

@Avish Saha

Some processors work in NiFi FlowFiles in batches. In your case one of the FlowFiles is failing to match on the regex which is causing a session roll back of the entire session. Also seeing as how the processor reports 8,000,000+ tasks in the last 5 minutes, it does not look like it is penalizing that one bad FlowFile. While all FlowFiles routed to a "failure" relationship are penalized, this is not true for all processors when a session rollback occurs.