question how i extract attribute from json file using nifi in Support Questions

how i extract attribute from json file using nifi

nejmhadjmbarek — Thu, 03 Mar 2016 22:10:26 GMT

I use gethttp processor to stream data using http request, so I need to get attribute from file json that I get from the first request and use it for another request, how I can do this with the existing nife processor.any help is appreciated

Re: how i extract attribute from json file using nifi

emaxwell — Thu, 03 Mar 2016 23:25:08 GMT

@nejm hadj You can parse a JSON with the EvaluateJsonPath processor. You will need to make sure that the information that your getHTTP processor outputs is strictly JSON, then feed it into EvaluateJsonPath. From there, you can reference the fields in the JSON and forward them on to other processors.

Twitter_Dashboard.xml is a Twitter processing template that has a good example of evaluating JSON files in it. Import the template into your NiFi dashboard then instantiate it to check out the EvaluateJsonPath processor to see how it's done:

Re: how i extract attribute from json file using nifi

Luos — Tue, 12 Apr 2016 07:48:43 GMT

@emaxwell

Hi emaxwell,

I have a same task to extract attributes from json file via gethttp with EvaluateJsonPath processor. However, the resulting data from the processor do not limit to the attributes, it pulled all attributes the original json data has. Is this meant to be so?

Thank you!

Shannon

Re: how i extract attribute from json file using nifi

emaxwell — Wed, 13 Apr 2016 03:14:22 GMT

@shannon luo

You can use an EvaluateJSONPath processor to pull out the fields that you want in the flow. You will create a parameter for each field in the JSON you wish to put on the output flow file. The name will be what you want the field to be called on the output, and the value will be an expression equating to the field in the input JSON (e.g. Name = twitter.name, Value = $.user.screen_name takes the input user:screen_name value from the JSON and creates a variable called twitter.name on the output flow file).

Thanks

Erik

Re: how i extract attribute from json file using nifi

Luos — Mon, 19 Aug 2019 11:39:23 GMT

@emaxwell

That was what I did. but still end up pulling all attribute. Going to Provenance, it showed EvaluateJsonPath pulled empty strings on all specified attribute. Would you be so kind an tell me where I got it wrong? -- I tried to pull "subject" property using value $.response.Matches.M

atch.Subject, $.Matches.Match.Subject, $.Match.Subject, and $.Subject. None of those worked. The source data looks like (see below):

Thanks !

Shannon

Re: how i extract attribute from json file using nifi

emaxwell — Mon, 19 Aug 2019 11:39:15 GMT

@shannon luo

What are the configuration attributes of your EvaluateJSONPath processor? Is you destination set to "flowfile-content" or "flowfile-attribute"? I have a processor set up to evaluate Twitter JSON, and the destination is set to "flowfile-attribute" with a number of attributes identified. Can you take a look at the attached image and see if your attributes are configured similarly?

Re: how i extract attribute from json file using nifi

Luos — Mon, 19 Aug 2019 11:39:07 GMT

@emaxwell

Yes, that was the template I used as the base for development. I used flowfile-attibute. Here is the screenshot for the properties. None of those properties added works.

Edit: May be this is what caused problem: Match has an array as its value. I tried $.response.Request.Actual.query, it worked fine. Do you know what to do to parse value from "Match"? (without going to a ExecuteScript).

Thanks so much for helping!

Shannon

Re: how i extract attribute from json file using nifi

randybovay — Mon, 12 Sep 2016 23:46:45 GMT

It would be nice to have a processor that specifically creates a list of attributes coming out of a JSON message.

Re: how i extract attribute from json file using nifi

sameer_khanwalk — Wed, 14 Sep 2016 19:41:22 GMT

Hi, I am converting the output of ExecuteSQL processor in JSON format and trying the above steps to extract some fetched column values from it. For that I have follow the same steps mentioned here to make it work but its not working. Plz suggest. My main idea to implement is to extract some column from executeSQL and based on that value I need to do some file ingestion. Help will be appreciated

Re: how i extract attribute from json file using nifi

kameswaran0631 — Sun, 18 Sep 2016 14:16:48 GMT

hi is there any way to extract the attributes without nifi ! i mean , i need to extract an attribute from a collection of tweet i already have !

Re: how i extract attribute from json file using nifi

bhagan — Fri, 30 Sep 2016 23:21:19 GMT

Try this:

ExecuteSQL > SplitAvro > ConvertAvrotoJSON > EvaluateJsonPath

SplitAvro creates individual Avro records

ConvertAvrotoJSON creates JSON from Avro

EvaluateJsonPath allows you to create new FlowFile attributes from JSON path.

Re: how i extract attribute from json file using nifi

seenu0104 — Tue, 08 Aug 2017 00:14:48 GMT

Hi, We have extracted the required fields from Json file as below.

It is not with only one processor. It is with two processors.

1. EvaluateJsonPath : What ever the configurations above mentioned are correct. This processor will validate whether this field exists or not. If not, then it will show that filed value as null.

2. AttributesToJSON : In the Attributes list property, give what ever the fields you want to pull (with comma separated) from the fields added in the EvaluateJsonPath processor.

[say you have added field1 to 6 in EvaluateJsonPath and then you want only field 1,3,4. Then give field1,field3,field4 in the Attributes list in the AttributesToJSON processor.]

Hope this helps you.

Re: how i extract attribute from json file using nifi

Hashem1980 — Wed, 20 Jul 2022 18:46:54 GMT

Hello, I have a json flow file as follows:

I used: EvaluateJsonPath:

But the result is: < Empty string set>

Thanks for your help and guide.

Re: how i extract attribute from json file using nifi

monmai — Wed, 17 Aug 2022 19:48:53 GMT

You are using an array, this is why you got null values. You have 2 options:

Change the expressions to $[0].ARRIVAL_DATE and $[0].BOOK_ID, and it will work
Read the file and split the array using the SplitJson processor - set JsonPath expression to: $, and connect the SplitJson processor to your EvaluateJsonPath processor and it should work.