question Re: NiFi - Using EvaluateJsonPath on flowfiles containing multiple records. in Support Questions

NiFi - Using EvaluateJsonPath on flowfiles containing multiple records.

Griggsy — Wed, 17 Nov 2021 16:06:01 GMT

Hello all

I have a NiFi data flow that has flowfiles containing multiple json records (with exactly the same received time) and I'm trying to extract the timestamp as a flowfile attribute as follows:

All the received times match as I'm using a PartitionRecord processor prior to EvaluateJsonPath, I have confirmed they are identical.

I am aware that I can split > extract > merge but I am trying to avoid that due to the high volume of data passing through this flow, is there any way to just extract the first timestamp for all the records in a single flowfile using EvaluateJsonPath? or another record processor?

I am also trying to avoid having to use ExtractText and regex to pull the timestamp, again due to the volume of data in the flow.

There is an existing flow working in our production environment using plain/text and regex etc. I am trying to re-design using record processing to improve efficiency.

any assistance welcome.

🙂

Re: NiFi - Using EvaluateJsonPath on flowfiles containing multiple records.

adhishankarit — Mon, 22 Nov 2021 09:46:42 GMT

Hi ,

Since there is no json structure mentioned in the question , Could you please check the JOLT nifi processor . Refer the jolt specification .

https://stackoverflow.com/questions/61697356/extract-first-element-array-with-jolt

Using Jolt nifi processor , we can perform many transformations from json file.

Thanks

Re: NiFi - Using EvaluateJsonPath on flowfiles containing multiple records.

Griggsy — Thu, 25 Nov 2021 11:18:36 GMT

Thanks for the reply, is there a way to use Jolt to extract to flowfile attribute?

Re: NiFi - Using EvaluateJsonPath on flowfiles containing multiple records.

MattWho — Thu, 02 Dec 2021 18:05:22 GMT

@Griggsy

If you can share a source json, it may help with more specific guidance in the community.

Thanks,
Matt