Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

nifi Json data using routeonattributeto to split attribute or attribute value

Solved Go to solution
Highlighted

nifi Json data using routeonattributeto to split attribute or attribute value

Explorer
I am currently working to consume data using Nifi related to tealium event stream and load into HDFS. Need help in filtering the data when source misses to send data for json attribute.
 

{"account":"newt","twitter:description":"Discover when your favorite New TV shows and hosts are being shown. ","og:locale":"en_US","dcterms:publisher":"New TV","original-source":"www.newtv.com/","og:url":"www.newtv.com/show/program-guide"}},"post_time":"2019-10-09 11:27:46","useragent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36","event_id":"12345"}

 

Sample above message. I am currently stuck with filtering the data when source misses to send data for event_id attribute.

 

Current Nifi flow, Consume Kafka -> Evaluate Json Path -> Jolttransform Json -> Evaluate Json Path-> RouteOnAttribute -> Merge Content -> Evaluate Json Path -> Update attribute -> PutHDFS ->MoveHDFS

 

Need help how to split data using RouteOnAttribute to split missing event_id or event_id value to two different flows.

Flow 1 - To split available event_id attribute.

Flow 2 - Missing Event_ID or Event_ID attribute values to treat as error and load into different flow.

2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: nifi Json data using routeonattributeto to split attribute or attribute value

Super Guru

In EvaluateJsonPath processor add new property to extract event_id value from the flowfile.

if flowfile is not having event_id then nifi adds empty value to the attribute.

EvaluateJsonPath Configs: enter image description here

Then by using RouteOnAttribute processor we can check the attribute value and route the flowfile accordingly.

RouteOnAttribute Configs:

not null value

${event_id:isEmpty():not()}

null value

${event_id:isEmpty()}

enter image description here

Then use null value and not null value relationships for further processing..!!

Re: nifi Json data using routeonattributeto to split attribute or attribute value

Explorer

@Shu_ashu Thank you for the solution.

I have got the issue resolved as it is working as expected.

2 REPLIES 2

Re: nifi Json data using routeonattributeto to split attribute or attribute value

Super Guru

In EvaluateJsonPath processor add new property to extract event_id value from the flowfile.

if flowfile is not having event_id then nifi adds empty value to the attribute.

EvaluateJsonPath Configs: enter image description here

Then by using RouteOnAttribute processor we can check the attribute value and route the flowfile accordingly.

RouteOnAttribute Configs:

not null value

${event_id:isEmpty():not()}

null value

${event_id:isEmpty()}

enter image description here

Then use null value and not null value relationships for further processing..!!

Re: nifi Json data using routeonattributeto to split attribute or attribute value

Explorer

@Shu_ashu Thank you for the solution.

I have got the issue resolved as it is working as expected.

Don't have an account?
Coming from Hortonworks? Activate your account here