Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to use an attribute in nifi to evaluate jsonpath from content of flowfile

Highlighted

How to use an attribute in nifi to evaluate jsonpath from content of flowfile

I use JSON Paths as attributes to nifi flowfiles (of JSON mime type). I want to evaluate these JSON Path over the content of the flowfiles but I can't use Evaluate JSON path processor as it does not support expression language. How can I do this as I can't hard code the JSON Paths in the processor, they are varying file to file.

4 REPLIES 4
Highlighted

Re: How to use an attribute in nifi to evaluate jsonpath from content of flowfile

Super Guru

You can accomplish this with ExecuteScript, the following example uses Groovy as the language and Jayway's JsonPath as the library for JSONPath parsing. First I had to download 3 JARs (json-path and its required transitive dependencies) into a directory:

16531-jsonpath-jars.png

Then I set the Module Directory property to the path of this directory:

16532-jsonpath-config.png

Note that I have also added a dynamic property, whose name will become an attribute and whose value supports Expression Language and (after evaluation) should contain a JSONPath expression used to retrieve the value from the content of the JSON flow file.

The Script Body is the following Groovy script:

import com.jayway.jsonpath.*
def flowFile = session.get()
if(!flowFile) return
def inputStream = session.read(flowFile)
def json = JsonPath.parse(inputStream)
inputStream.close()
context.properties.findAll {p,s -> p.dynamic}.each {pd, name ->
   def prop = context.getProperty(pd)
   try {
      flowFile = session.putAttribute(flowFile, pd.name, json.read(prop.evaluateAttributeExpressions(flowFile).value))
   } catch (e) {
      log.error("Error evaluating JSONPath expression in property $name: ${prop?.value} , ignoring...", e)
   }
}
session.transfer(flowFile, REL_SUCCESS)

I tested this with a GenerateFlowFile:

16533-jsonpath-gff-config.png

After the ExecuteScript transfers the flow file, it has the desired attribute name/value:

16534-jsonpath-result.png

This should work with any number of attributes/JSONPaths per flow file. In addition, I have written NIFI-4100 to cover the improvement to the EvaluateJsonPath processor to support Expression Language.

Highlighted

Re: How to use an attribute in nifi to evaluate jsonpath from content of flowfile

New Contributor

There is such a scene,an attribute IP address in the flowfile,I have to call a Python function,to resolve the region infomation through IP address,then add the region infomation to new flowfile,how should I do? Thanks a lot for replying.@Matt Burgess

Highlighted

Re: How to use an attribute in nifi to evaluate jsonpath from content of flowfile

New Contributor

16551-hehehe.jpg

I don't know why I can't ask question

Re: How to use an attribute in nifi to evaluate jsonpath from content of flowfile

New Contributor

Hi @mburgess I tried the solution but it is not working for other JSON paths like $.key

I tried multiple JSON paths but seems like it only works for $.cells.year. Please let me know your thoughts. I followed the exact steps you provided.

Don't have an account?
Coming from Hortonworks? Activate your account here