Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Manipulating the keys and/or values of a nested JSON

avatar
Explorer
Hey guys,

is there a way of using existing NiFi processors to manipulate nested JSON-files that are mainly of the same structure and have like 10k+ lines? I´ve read a lot About the ReplaceText processor (Maybe JoltTransformJson?), but I could not get it to work. The main Problem is that I have to either change the key or the value, or sometimes even both.

So I am currently having a Model and am receiving data that has to get mapped to this. The only constant is the structure of the json and the keys. So there should be a way to transform them into the key names that are required.

{"id":"test"},{"id":"14"}

For example, if I get this JSON, I would have to transform "id" to "number" and "test" to "3", for example.

Is there a way of doing so without creating a new processor?

Sincerely,

J

1 ACCEPTED SOLUTION

avatar
Rising Star
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
5 REPLIES 5

avatar
Explorer

avatar
Rising Star
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Explorer

@Prabhu M Thanks for your Reply, I am really appreciating it. Is there a way - by using Json Jsolt - to go through the nested Array and differentiate between the same called keys in different nestings? for example there is an id directly in the data Array and an id in another Array that is inside the data Array?

Additionally, is there a possible way of using ExecuteScript? I tried something like this:

But I could not get it to work properly. This seems like the easiest attempt to me, but I do not know how to set/write a new key with the old value.

import org.apache.commons.io.IOUtils
import java.nio.charset.StandardCharsets
import groovy.json.*
import java.lang.*
import java.util.*
import java.io.*
import java.net.*
import groovy.lang.*
import groovy.util.*
import java.math.BigInteger
import java.math.BigDecimal
import testST.Deg2UTM
def flowFile = session.get()
if (!flowFile) return
 
try {
  flowFile = session.write(flowFile,
      { inputStream, outputStream ->
          def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
          def obj = new JsonSlurper().parseText(text)
 
          // Updade postal_code to PLZ and add the former value
    for(i in 0..obj.data.size()-1){
   def value = obj.data[0].remove("postal_code")
   obj.data[0].["PLZ"]=value
    }
          // Output updated JSON
          def json = JsonOutput.toJson(obj)
          outputStream.write(JsonOutput.prettyPrint(json).getBytes(StandardCharsets.UTF_8))
      } as StreamCallback)
  flowFile = session.putAttribute(flowFile, "filename", flowFile.getAttribute('filename').tokenize('.')[0]+'_translated.json')
  session.transfer(flowFile, REL_SUCCESS)
 
} catch(Exception e) {
  log.error('Error during JSON operations', e)
  session.transfer(flowFile, REL_FAILURE)
}


<br>

avatar
Master Guru

Yes you can access the same key names at different places in the JSON. Can you provide a sample input JSON and the expected output JSON?

avatar
Explorer

Hey @Matt Burgess , thanks for your reply. I got this issue solved. My new problem is explained in a new topic. I get for every Table a new Json that I have to match with the self-generated Json and put it in the DB with mutliple tables. And I do not know how to achieve this in a proper way.