Skip to main content
kekotron
Silver ⭐️
February 12, 2024
Solved

Replacing characters within a nested json

Hello,

I have nested JSON objects coming in a flowfile, looking like this (example):

 

{ "a" : 1, "b" : {"asd" : 156, "bed" : "lalala"}, "c" : "abba", "d" : {"add" : 872, "bla" : "bebebe"} }
 
I'd like to put double quotes around every nested json object and also escape any double quote inside of that nested json. The result would look like this:
 
{ "a" : 1, "b" : "{\"asd\" : 156, \"bed\" : \"lalala\"}", "c" : "abba", "d" : "{\"add\" : 872, \"bla\" : \"bebebe\"}" }
 
I tried doing a replacetext with a regex like this: (?<=:.*\{[^}]*)"
However this is utilizing a negative lookbehind with varying length and it seems this isn't supported in java/nifi. What would be the best approach here? Thanks!
SAMSAL
SAMSALAnswer
Platinum ⭐️⭐️⭐️
February 13, 2024

Hi @kekotron ,

There is no simple out of the box solution for this that I can think of. The easiest way is to use ExecuteScript processor that parse the json  as a map, then loop through each key and check if the value of that key is of type map as well- which means nested json - to then convert the map to json string and re assign back to the same key.

The ExecuteScript below is written using groovy but you can do the same with other languages as well.

import org.apache.commons.io.IOUtils
import java.nio.charset.StandardCharsets
import groovy.json.JsonOutput
import groovy.json.JsonSlurper

flowFile = session.get()
if(!flowFile) return

// Cast a closure with an inputStream and outputStream parameter to StreamCallback
flowFile = session.write(flowFile, {inputStream, outputStream ->
jsonText = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

jsonMap = new JsonSlurper().parseText(jsonText)

jsonMap.each{k,v->
if(jsonMap[k] instanceof Map)
jsonMap[k] = JsonOutput.toJson(jsonMap[k])

}

outputStream.write(JsonOutput.toJson(jsonMap).getBytes(StandardCharsets.UTF_8))
} as StreamCallback)
session.transfer(flowFile, REL_SUCCESS)

For more info on how to write write script inside ExecuteScript:

https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-2/ta-p/249018

If you find this helpful please accept solution.

Thanks

 

3 replies

DianaTorres
Community Manager
Community Manager
February 13, 2024

@kekotron Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @MattWho @joseomjr  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
SAMSAL
SAMSALAnswer
Platinum ⭐️⭐️⭐️
February 13, 2024

Hi @kekotron ,

There is no simple out of the box solution for this that I can think of. The easiest way is to use ExecuteScript processor that parse the json  as a map, then loop through each key and check if the value of that key is of type map as well- which means nested json - to then convert the map to json string and re assign back to the same key.

The ExecuteScript below is written using groovy but you can do the same with other languages as well.

import org.apache.commons.io.IOUtils
import java.nio.charset.StandardCharsets
import groovy.json.JsonOutput
import groovy.json.JsonSlurper

flowFile = session.get()
if(!flowFile) return

// Cast a closure with an inputStream and outputStream parameter to StreamCallback
flowFile = session.write(flowFile, {inputStream, outputStream ->
jsonText = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

jsonMap = new JsonSlurper().parseText(jsonText)

jsonMap.each{k,v->
if(jsonMap[k] instanceof Map)
jsonMap[k] = JsonOutput.toJson(jsonMap[k])

}

outputStream.write(JsonOutput.toJson(jsonMap).getBytes(StandardCharsets.UTF_8))
} as StreamCallback)
session.transfer(flowFile, REL_SUCCESS)

For more info on how to write write script inside ExecuteScript:

https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-2/ta-p/249018

If you find this helpful please accept solution.

Thanks

 

Translate
kekotron
kekotronAuthor
Silver ⭐️
February 13, 2024

Thanks! This scripting approach looks to be working. I'll be sure to look into the ExecuteScript processor more 🙂

Translate