How to load attributes from json

[#1]

Many ppl has me this question on how to load the attributes from json.

[#2]

Load the json in to flowfile content . Feed this to a Executescript Processor with the below code.

Note this code assume that the json does not have nested element. Hope this help

import org.apache.commons.io.IOUtils
import java.nio.charset.*
def flowFile = session.get();
if (flowFile == null) {
    return;
}
def slurper = new groovy.json.JsonSlurper()
def attrs = [:] as Map<String,String>
session.read(flowFile,
    { inputStream ->
        def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
        def obj = slurper.parseText(text)
        obj.each {k,v ->
           attrs[k] = v.toString()
        }
    } as InputStreamCallback)

def text = ''
// Cast a closure with an outputStream parameter to OutputStreamCallback
flowFile = session.write(flowFile, {outputStream ->
  outputStream.write(text.getBytes(StandardCharsets.UTF_8))
} as OutputStreamCallback)
flowFile = session.putAllAttributes(flowFile, attrs)
session.transfer(flowFile , REL_SUCCESS)

[#3]

@tthomas Have you consider EvaluateJsonPath with Destination flowfile attribute? https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

Here is an example screenshot of how it looks for twitter json feed:

HTH

[#4]

@tthomas If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

[#5]

Hi @tthomas

You can use EvaluateJsonPath to extract a JSON field and add it as a flow file attribute : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

If your JSON is the following and you want to add a flow file attribute called timestamp

{
“created_at” : “Thu Sep 28 08:08:09 CEST 2017”,
“id_store” : 4,
“event_type” : “store capacity”,
“id_transaction” : “1009331737896598289”,
“id_product” : 889,
“value_product” : 45
}

you can add an EvaluateJsonPath and add an attribute timestamp with the value $.created_at

[#6]

Abdelkrim Hadjidj, yes you can do it if you know what you want to extract. This code will help if user want to load the attribute from json file , so that the attribute value is not hardcoded in the flow.xml. Often some values can be kept in variable for specific environment , eg : Dev , Test , Prod. and these can be separated out in a json file which will not change with the updates in the flow.xml

With the latest version of nifi (variable registry ) this is not required. My intention is just to show the need for the same.