Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to load attributes from json

avatar
Contributor

Many ppl has me this question on how to load the attributes from json.

5 REPLIES 5

avatar
Contributor

Load the json in to flowfile content . Feed this to a Executescript Processor with the below code.

Note this code assume that the json does not have nested element. Hope this help

import org.apache.commons.io.IOUtils
import java.nio.charset.*
def flowFile = session.get();
if (flowFile == null) {
    return;
}
def slurper = new groovy.json.JsonSlurper()
def attrs = [:] as Map<String,String>
session.read(flowFile,
    { inputStream ->
        def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
        def obj = slurper.parseText(text)
        obj.each {k,v ->
           attrs[k] = v.toString()
        }
    } as InputStreamCallback)

def text = ''
// Cast a closure with an outputStream parameter to OutputStreamCallback
flowFile = session.write(flowFile, {outputStream ->
  outputStream.write(text.getBytes(StandardCharsets.UTF_8))
} as OutputStreamCallback)
flowFile = session.putAllAttributes(flowFile, attrs)
session.transfer(flowFile , REL_SUCCESS)

avatar

@tthomas Have you consider EvaluateJsonPath with Destination flowfile attribute? https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

Here is an example screenshot of how it looks for twitter json feed:

76579-screen-shot-2018-06-04-at-14756-pm.png

HTH

avatar

@tthomas If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

avatar

Hi @tthomas

You can use EvaluateJsonPath to extract a JSON field and add it as a flow file attribute : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

If your JSON is the following and you want to add a flow file attribute called timestamp

{
“created_at” : “Thu Sep 28 08:08:09 CEST 2017”,
“id_store” : 4,
“event_type” : “store capacity”,
“id_transaction” : “1009331737896598289”,
“id_product” : 889,
“value_product” : 45
}

you can add an EvaluateJsonPath and add an attribute timestamp with the value $.created_at

avatar
Contributor

Abdelkrim Hadjidj, yes you can do it if you know what you want to extract. This code will help if user want to load the attribute from json file , so that the attribute value is not hardcoded in the flow.xml. Often some values can be kept in variable for specific environment , eg : Dev , Test , Prod. and these can be separated out in a json file which will not change with the updates in the flow.xml

With the latest version of nifi (variable registry ) this is not required. My intention is just to show the need for the same.