Support Questions
Find answers, ask questions, and share your expertise

How to use an attribute in nifi to evaluate jsonpath from content of flowfile

I use JSON Paths as attributes to nifi flowfiles (of JSON mime type). I want to evaluate these JSON Path over the content of the flowfiles but I can't use Evaluate JSON path processor as it does not support expression language. How can I do this as I can't hard code the JSON Paths in the processor, they are varying file to file.

4 REPLIES 4

Super Guru

You can accomplish this with ExecuteScript, the following example uses Groovy as the language and Jayway's JsonPath as the library for JSONPath parsing. First I had to download 3 JARs (json-path and its required transitive dependencies) into a directory:

16531-jsonpath-jars.png

Then I set the Module Directory property to the path of this directory:

16532-jsonpath-config.png

Note that I have also added a dynamic property, whose name will become an attribute and whose value supports Expression Language and (after evaluation) should contain a JSONPath expression used to retrieve the value from the content of the JSON flow file.

The Script Body is the following Groovy script:

import com.jayway.jsonpath.*
def flowFile = session.get()
if(!flowFile) return
def inputStream = session.read(flowFile)
def json = JsonPath.parse(inputStream)
inputStream.close()
context.properties.findAll {p,s -> p.dynamic}.each {pd, name ->
   def prop = context.getProperty(pd)
   try {
      flowFile = session.putAttribute(flowFile, pd.name, json.read(prop.evaluateAttributeExpressions(flowFile).value))
   } catch (e) {
      log.error("Error evaluating JSONPath expression in property $name: ${prop?.value} , ignoring...", e)
   }
}
session.transfer(flowFile, REL_SUCCESS)

I tested this with a GenerateFlowFile:

16533-jsonpath-gff-config.png

After the ExecuteScript transfers the flow file, it has the desired attribute name/value:

16534-jsonpath-result.png

This should work with any number of attributes/JSONPaths per flow file. In addition, I have written NIFI-4100 to cover the improvement to the EvaluateJsonPath processor to support Expression Language.

New Contributor

There is such a scene,an attribute IP address in the flowfile,I have to call a Python function,to resolve the region infomation through IP address,then add the region infomation to new flowfile,how should I do? Thanks a lot for replying.@Matt Burgess

New Contributor

16551-hehehe.jpg

I don't know why I can't ask question

New Contributor

Hi @mburgess I tried the solution but it is not working for other JSON paths like $.key

I tried multiple JSON paths but seems like it only works for $.cells.year. Please let me know your thoughts. I followed the exact steps you provided.

; ;