Create a json payload using a template and its values

scoutjohn — Fri, 19 Jul 2024 14:27:24 GMT

I have a python script that takes in text input in the format like this

{"order":{"Testorder":{"operation":"create","specification":{"name":"${name}","version":"1.0.0-SNAPSHOT"},"parameters":{"controllerName":"${controllername}","parameters":{"parameter1":"${value1}","parameter2":"${value2}"}}}}} {"name": "TestService", "controllername": "MyController_d","value1": "test","value2": "speed"}

First line is a JSON Template with place holders. Second line another JSON string which holds the value to be replaced in the first line. (whole input (first line and second line together) are not a valid JSON)

It creates an output like this

{ "order" : { "Testorder" : { "operation" : "create", "specification" : { "name" : "TestService", "version" : "1.0.2-SNAPSHOT" }, "parameters" : { "controllerName" : "MyController_d", "parameters" : { "parameter1" : "test", "parameter2" : "strength" } } } } }

I have implemented this using a python script inside ExecuteScript processor

import json import sys import traceback from java.nio.charset import StandardCharsets from org.apache.commons.io import IOUtils from org.apache.nifi.processor.io import StreamCallback from string import Template class PyStreamCallback(StreamCallback): def __init__(self): pass def process(self, inputStream, outputStream): try: input_text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) incoming_flow = input_text.split('\n', 1) if len(incoming_flow) < 2: raise ValueError("Input data does not contain both template and values.") template_str, values_str = incoming_flow template_str = template_str.strip() values_str = values_str.strip() json.loads(template_str) parameter_obj = json.loads(values_str) tpl = Template(template_str) json_string = tpl.substitute(parameter_obj) # Replace placeholders and unwanted characters replacements = { "u'": "'", 'u"': '"', '"{': '{', '}"': '}', "'": '"', "False": "false", "True": "true", '"[': "[", ']"': "]" } for old, new in replacements.items(): json_string = json_string.replace(old, new) outputStream.write(bytearray(json_string.encode('utf-8'))) except Exception as e: traceback.print_exc(file=sys.stdout) raise e flowFile = session.get() if flowFile is not None: try: flowFile = session.write(flowFile, PyStreamCallback()) session.putAttribute(flowFile, 'mime.type', 'application/json') session.transfer(flowFile, REL_SUCCESS) except ValueError as ve: session.putAttribute(flowFile, 'error_val', str({"Script.Exception": str(ve)})) session.transfer(flowFile, REL_FAILURE) except Exception as e: session.putAttribute(flowFile, 'error_val', str({"Script.Exception": str(e)})) session.transfer(flowFile, REL_FAILURE)

Though there are a lot of string manipulation, so far it is working fine.

Since python is deprecated in the ExecuteScript Processor. Was wondering if it would be possible to do this same operations using any other processor in NiFi without writing any of custom script.

we're using 1.24.0 version of NiFi.

Can the ReplaceText processor be used to achieve this or is there any other recommendation?

Re: Create a json payload using a template and its values

joseomjr — Mon, 22 Jul 2024 19:44:24 GMT

You could implement this using Groovy script which is forward and backward compatible. I've taken code written in NiFi 1.X and works just fine in 2.X

Re: Create a json payload using a template and its values

scoutjohn — Thu, 07 Nov 2024 06:47:45 GMT

@joseomjr , Thank you for responding,

I instead chose a hard way approach. I thought why not create a custom Nar, which takes in 2 parameters, 1 for the json template with placeholders and 2 with its respective values.

used sort of a recursion to create the final output.

for and input like

Template

{"details":{"name":"${name}","age":"${age}","superpower":"${superpower}"}}

value :

{"name": "Clark Kent", "age": "35","superpower": "Superman"}

gives output as

{
"details": {
"name": "Clark Kent",
"age": "35",
"superpower": "Superman"
}
}

question Re: Create a json payload using a template and its values in Support Questions

Create a json payload using a template and its values

Re: Create a json payload using a template and its values

Re: Create a json payload using a template and its values