Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to use an attribute in nifi to evaluate jsonpath from content of flowfile

How to use an attribute in nifi to evaluate jsonpath from content of flowfile

New Contributor

I use JSON Paths as attributes to nifi flowfiles (of JSON mime type). I want to evaluate these JSON Path over the content of the flowfiles but I can't use Evaluate JSON path processor as it does not support expression language. How can I do this as I can't hard code the JSON Paths in the processor, they are varying file to file.

3 REPLIES 3

Re: How to use an attribute in nifi to evaluate jsonpath from content of flowfile

You can accomplish this with ExecuteScript, the following example uses Groovy as the language and Jayway's JsonPath as the library for JSONPath parsing. First I had to download 3 JARs (json-path and its required transitive dependencies) into a directory:

16531-jsonpath-jars.png

Then I set the Module Directory property to the path of this directory:

16532-jsonpath-config.png

Note that I have also added a dynamic property, whose name will become an attribute and whose value supports Expression Language and (after evaluation) should contain a JSONPath expression used to retrieve the value from the content of the JSON flow file.

The Script Body is the following Groovy script:

import com.jayway.jsonpath.*
def flowFile = session.get()
if(!flowFile) return
def inputStream = session.read(flowFile)
def json = JsonPath.parse(inputStream)
inputStream.close()
context.properties.findAll {p,s -> p.dynamic}.each {pd, name ->
   def prop = context.getProperty(pd)
   try {
      flowFile = session.putAttribute(flowFile, pd.name, json.read(prop.evaluateAttributeExpressions(flowFile).value))
   } catch (e) {
      log.error("Error evaluating JSONPath expression in property $name: ${prop?.value} , ignoring...", e)
   }
}
session.transfer(flowFile, REL_SUCCESS)

I tested this with a GenerateFlowFile:

16533-jsonpath-gff-config.png

After the ExecuteScript transfers the flow file, it has the desired attribute name/value:

16534-jsonpath-result.png

This should work with any number of attributes/JSONPaths per flow file. In addition, I have written NIFI-4100 to cover the improvement to the EvaluateJsonPath processor to support Expression Language.

Re: How to use an attribute in nifi to evaluate jsonpath from content of flowfile

New Contributor

There is such a scene,an attribute IP address in the flowfile,I have to call a Python function,to resolve the region infomation through IP address,then add the region infomation to new flowfile,how should I do? Thanks a lot for replying.@Matt Burgess

Re: How to use an attribute in nifi to evaluate jsonpath from content of flowfile

New Contributor

16551-hehehe.jpg

I don't know why I can't ask question

Don't have an account?
Coming from Hortonworks? Activate your account here