Add Flow file string contet to json value with out using ExtractText processor

[#1]

Hi

I have a flow file with huge string content and I want to create a JSON and add this string to the JSON as a value with a static key.

Example

Flow file string content: "Example string"

Needed output :{"Static key":"Example string"}

I tried to add the string content to the flow file attribute using ExtractText and use attribute to JSON processor to generate the needed output but I have 2 issues with this approach:-

1- The string content size is not under my control and I dont know how much I need to configure the Maximum Buffer Size and Maximum Capture Group Length.

2- Adding the content to the flow file attribute will duplicate the flow file size and this will impact the performance.

3- Finally I feel it is a workaround solution :))

Is there any other way to solve this issue?

[#2]

Hi the only way I can think of to avoid duplicating the content of the flowfile and not having to worry about the Maximum Buffer Size in the ExtractText processor is to use the ExecuteScript processor to update the flowfile content into json format. Here is an example of Script using "ECMAScript" in the ExecuteScript Processor "Script Body" property that can do that:

var StreamCallback =  Java.type("org.apache.nifi.processor.io.StreamCallback");
var IOUtils = Java.type("org.apache.commons.io.IOUtils");
var StandardCharsets = Java.type("java.nio.charset.StandardCharsets");

var flowFile = session.get();
if(flowFile != null) {
  // Create a new StreamCallback, passing in a function to define the interface method
  flowFile = session.write(flowFile,
    new StreamCallback(function(inputStream, outputStream) {
        var text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
		var obj = new Object();
		 obj.SomeKey = text;
		 var jsonString= JSON.stringify(obj);
        outputStream.write(jsonString.getBytes(StandardCharsets.UTF_8))
    }));

}
session.transfer(flowFile, REL_SUCCESS)

Hope that helps. If it does please Accept Solution. Thanks

[#3]

@KhASQ ,

Besides @SAMSAL solution, you can also use ReplaceText to eliminate the need of extracting the entire content as an attribute. You'd still have to set a large enough buffer, though, to ensure your largest message could be processed.

Cheers,

André