Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

nifi Json data using routeonattributeto to split attribute or attribute value

Explorer
I am currently working to consume data using Nifi related to tealium event stream and load into HDFS. Need help in filtering the data when source misses to send data for json attribute.
 

{"account":"newt","twitter:description":"Discover when your favorite New TV shows and hosts are being shown. ","og:locale":"en_US","dcterms:publisher":"New TV","original-source":"www.newtv.com/","og:url":"www.newtv.com/show/program-guide"}},"post_time":"2019-10-09 11:27:46","useragent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36","event_id":"12345"}

 

Sample above message. I am currently stuck with filtering the data when source misses to send data for event_id attribute.

 

Current Nifi flow, Consume Kafka -> Evaluate Json Path -> Jolttransform Json -> Evaluate Json Path-> RouteOnAttribute -> Merge Content -> Evaluate Json Path -> Update attribute -> PutHDFS ->MoveHDFS

 

Need help how to split data using RouteOnAttribute to split missing event_id or event_id value to two different flows.

Flow 1 - To split available event_id attribute.

Flow 2 - Missing Event_ID or Event_ID attribute values to treat as error and load into different flow.

2 ACCEPTED SOLUTIONS

Accepted Solutions

Super Guru

In EvaluateJsonPath processor add new property to extract event_id value from the flowfile.

if flowfile is not having event_id then nifi adds empty value to the attribute.

EvaluateJsonPath Configs: enter image description here

Then by using RouteOnAttribute processor we can check the attribute value and route the flowfile accordingly.

RouteOnAttribute Configs:

not null value

${event_id:isEmpty():not()}

null value

${event_id:isEmpty()}

enter image description here

Then use null value and not null value relationships for further processing..!!

View solution in original post

Explorer

@Shu_ashu Thank you for the solution.

I have got the issue resolved as it is working as expected.

View solution in original post

2 REPLIES 2

Super Guru

In EvaluateJsonPath processor add new property to extract event_id value from the flowfile.

if flowfile is not having event_id then nifi adds empty value to the attribute.

EvaluateJsonPath Configs: enter image description here

Then by using RouteOnAttribute processor we can check the attribute value and route the flowfile accordingly.

RouteOnAttribute Configs:

not null value

${event_id:isEmpty():not()}

null value

${event_id:isEmpty()}

enter image description here

Then use null value and not null value relationships for further processing..!!

View solution in original post

Explorer

@Shu_ashu Thank you for the solution.

I have got the issue resolved as it is working as expected.

View solution in original post