question NiFi some basics in Archives of Support Questions (Read Only)

NiFi some basics

avijeetd — Mon, 28 Nov 2016 18:00:26 GMT

Hi,

I am using a InvokeHTTP to call a external rest api and then Extract a few values from the JSON response using EvaluateJSONPath

What I am not able to understand is, why are we doing the extract from flowfile-attributes and not flowfile-content

Comparing the terminology with flume (which I have used before), flowfile-attributes are the Event headers and flowfile-content is Event body

Then how is EvaluateJSONPath is extracting from header using EL

Thanks,

Avijeet

Re: NiFi some basics

gkeys — Fri, 02 Dec 2016 20:04:37 GMT

Please note that the EvaluateJsonPath

extracts from content and
places to content or attributes (depending on whether you configure the Destination property as flowfile-content or flowfile-attribute)

Thus it extracts from content and places to content or attributes.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.EvaluateJsonPath/index.html

If you want to extract the headers from the response of InvokeHTTPRequest, you will use EvaluateJsonPath as described above, since the HTTP response header will be part of the json returned and thus part of the FlowFile content. The following post may be helpful:

https://community.hortonworks.com/questions/21011/how-i-extract-attribute-from-json-file-using-nifi.html

If this answers your question, let me know by accepting the answer; else, let me know of any gaps or follow-up questions.

Re: NiFi some basics

avijeetd — Fri, 02 Dec 2016 20:45:17 GMT

Thanks @Greg Keys - I was under impression the we are extracting from attributes. Can you pls share any documentation which clearly explain the lifecycle of a Event/flowfile going through a dataflow and any best practices.

Re: NiFi some basics

gkeys — Fri, 02 Dec 2016 21:05:39 GMT

This explains flowfiles:

[basic] http://docs.hortonworks.com/HDPDocuments/HDF1/HDF-1.1.1/bk_UserGuide/content/terminology.html
[details/flowfile lifecycle] https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html#DeeperView

This is a good overview of NiFI: https://nifi.apache.org/docs/nifi-docs/html/getting-started.html.

Each processor follows the same idea of flowfile = content + attributes. Following the links to each shows more specialized behavior for that processor in operating on flowfiles (which are passed from one processor to the next via connections)