Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

How to load attributes from json


Many ppl has me this question on how to load the attributes from json.



Load the json in to flowfile content . Feed this to a Executescript Processor with the below code.

Note this code assume that the json does not have nested element. Hope this help

import java.nio.charset.*
def flowFile = session.get();
if (flowFile == null) {
def slurper = new groovy.json.JsonSlurper()
def attrs = [:] as Map<String,String>,
    { inputStream ->
        def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
        def obj = slurper.parseText(text)
        obj.each {k,v ->
           attrs[k] = v.toString()
    } as InputStreamCallback)

def text = ''
// Cast a closure with an outputStream parameter to OutputStreamCallback
flowFile = session.write(flowFile, {outputStream ->
} as OutputStreamCallback)
flowFile = session.putAllAttributes(flowFile, attrs)
session.transfer(flowFile , REL_SUCCESS)

@tthomas Have you consider EvaluateJsonPath with Destination flowfile attribute?

Here is an example screenshot of how it looks for twitter json feed:



@tthomas If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

Hi @tthomas

You can use EvaluateJsonPath to extract a JSON field and add it as a flow file attribute :

If your JSON is the following and you want to add a flow file attribute called timestamp

“created_at” : “Thu Sep 28 08:08:09 CEST 2017”,
“id_store” : 4,
“event_type” : “store capacity”,
“id_transaction” : “1009331737896598289”,
“id_product” : 889,
“value_product” : 45

you can add an EvaluateJsonPath and add an attribute timestamp with the value $.created_at


Abdelkrim Hadjidj, yes you can do it if you know what you want to extract. This code will help if user want to load the attribute from json file , so that the attribute value is not hardcoded in the flow.xml. Often some values can be kept in variable for specific environment , eg : Dev , Test , Prod. and these can be separated out in a json file which will not change with the updates in the flow.xml

With the latest version of nifi (variable registry ) this is not required. My intention is just to show the need for the same.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.