Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Merge multiple json with the same id in Nifi?

avatar
New Contributor

I have this flowchart in nifi, where json files (ConsumerKafka) are consumed and then pass them in mongodb

 

the first Json file is:

 

{"_id": "Eems_20200312", "Name": "Eems", "Date": "2020-03-12 23:14:02", "Volatility ": "5,328", "Value risk": "12,395"}
{"_id": " Esprinet_20200312", "Name": " Esprinet ", "Date": "2020-03-12 23:14:02", "Volatility ": "5,654", "Value risk": "12,765"}
{"_id": " Eurotech_20200312", "Name": " Eurotech ", "Date": "2020-03-12 23:14:02", "Volatility ": "5,534", "Value risk": "12,832"}

 

the second Json file is:

 

{"_id": "Eems_20200312", "Name": "Eems", "Last": "0,066", "%": "-9,04", "Hour": "17:26", "Min": "0,061", "Max": "0,069", "Open": "0,068", "Date": "2020-03-12 23:14:02"}
{"_id": "Esprinet_20200312", "Name": "Esprinet", "Last": "3,495", "%": "-14,86", "Hour": "17:35", "Min": "3,47", "Max": "3,89", "Open": "3,89", "Date": "2020-03-12 23:14:02"}
{"_id": "Eurotech_20200312", "Name": "Eurotech", "Last": "5,05", "%": "-12,17", "Hour": "17:36", "Min": "5,05", "Max": "5,56", "Open": "5,50", "Date": "2020-03-12 23:14:02"}


The result I would like in Mongodb is this:

 

{"_id": "Eems_20200312", "Name": "Eems", "Last": "0,066", "%": "-9,04", "Hour": "17:26", "Min": "0,061", "Max": "0,069", "Open": "0,068", "Date": "2020-03-12 23:14:02", “Volatility”: “5,328” , “Value Risk”: “12,395”}
{"_id": "Esprinet_20200312", "Name": "Esprinet", "Last": "3,495", "%": "-14,86", "Hour": "17:35", "Min": "3,47", "Max": "3,89", "Open": "3,89", "Date": "2020-03-12 23:14:02", "Volatility ": "5,654", "Value risk": "12,765"}
{"_id": "Eurotech_20200312", "Name": "Eurotech", "Last": "5,05", "%": "-12,17", "Hour": "17:36", "Min": "5,05", "Max": "5,56", "Open": "5,50", "Date": "2020-03-12 23:14:02"", "Volatility ": "5,534", "Value risk": "12,832"}

 

How can I do it using Nifi?

I would like to merge all the content when the primary key is the same and would like to know if the flow chart is correct or if i need to add something else.

 

1 ACCEPTED SOLUTION

avatar
Explorer
2 REPLIES 2

avatar
Explorer

What you are trying to achieve is can be done using NiFi Lookup Processor. Please explore more on that.

avatar
Explorer