NiFi some basics

[#1]

Hi,

I am using a InvokeHTTP to call a external rest api and then Extract a few values from the JSON response using EvaluateJSONPath

What I am not able to understand is, why are we doing the extract from flowfile-attributes and not flowfile-content

Comparing the terminology with flume (which I have used before), flowfile-attributes are the Event headers and flowfile-content is Event body

Then how is EvaluateJSONPath is extracting from header using EL

Thanks,

Avijeet

[#2 · Accepted solution]

Please note that the EvaluateJsonPath

• extracts from content and
• places to content or attributes (depending on whether you configure the Destination property as flowfile-content or flowfile-attribute)

Thus it extracts from content and places to content or attributes.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.EvaluateJsonPa...

If you want to extract the headers from the response of InvokeHTTPRequest, you will use EvaluateJsonPath as described above, since the HTTP response header will be part of the json returned and thus part of the FlowFile content. The following post may be helpful:

https://community.hortonworks.com/questions/21011/how-i-extract-attribute-from-json-file-using-nifi....

If this answers your question, let me know by accepting the answer; else, let me know of any gaps or follow-up questions.

[#3]

Thanks @Greg Keys - I was under impression the we are extracting from attributes. Can you pls share any documentation which clearly explain the lifecycle of a Event/flowfile going through a dataflow and any best practices.

[#4]

This explains flowfiles:

• [basic] http://docs.hortonworks.com/HDPDocuments/HDF1/HDF-1.1.1/bk_UserGuide/content/terminology.html
• [details/flowfile lifecycle] https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html#DeeperView

This is a good overview of NiFI: https://nifi.apache.org/docs/nifi-docs/html/getting-started.html.

Each processor follows the same idea of flowfile = content + attributes. Following the links to each shows more specialized behavior for that processor in operating on flowfiles (which are passed from one processor to the next via connections)