Support Questions

Find answers, ask questions, and share your expertise

How to filter the incoming JSON payload based on the values in flow file content using NiFi?

avatar
Rising Star

Hi,

I have the incoming flowfiles with the json content which have multiple event types.

I need to filter only certain events based on two of the JSON key/value.

Sample input data:

{"eventType":"Mobile","timestamp":1551280374552,"name":"ERROR"}
{"eventType":"Mobile","timestamp":1551280374552,"name":"APP_START"}
{"eventType":"Immobile","timestamp":1551280374552,"name":"ERROR"}
{"eventType":"Immobile","timestamp":1551280374552,"name":"ERROR"}
{"eventType":"Mobile","timestamp":1551280374552,"name":"PLAYBACK_ERROR"}
{"eventType":"Mobile","timestamp":1551280374552,"name":"Other"}
{"eventType":"MobileCrash","timestamp":1551280374552,"name":"ERROR"}

Filter Condition to be applied :

Flowfile content having

If eventtype = "Mobile" and name = (ERROR or APP_START or PLAYBACK_ERROR) or

eventtype = "MobileCrash"


Also, after the filter i have to rename the json key as,

eventType = vendorEventType

timestamp = currentTimestamp

name = someName


Desired Output:

{"vendorEventType":"Mobile","currentTimestamp":1551280374552,"someName":"ERROR"}
{"vendorEventType":"Mobile","currentTimestamp":1551280374552,"someName":"APP_START"}{"vendorEventType":"Mobile","currentTimestamp":1551280374552,"someName":"PLAYBACK_ERROR"}{"vendorEventType":"MobileCrash","currentTimestamp":1551280374552,"someName":"ERROR"}


Please help.


1 REPLY 1

avatar
Master Guru

@Bala S

Use QueryRecord processor with Record Reader and Record Writer controller services, and QueryRecord processor will result out the records that matches your Query.

Use ApacheCalcite sql syntax to filter out the required records from the flowfile content.

Some useful links link1, link2