Merge multiple json with the same id in Nifi?

NifiMongoDB — Sat, 14 Mar 2020 17:00:35 GMT

I have this flowchart in nifi, where json files (ConsumerKafka) are consumed and then pass them in mongodb

the first Json file is:

{"_id": "Eems_20200312", "Name": "Eems", "Date": "2020-03-12 23:14:02", "Volatility ": "5,328", "Value risk": "12,395"}
{"_id": " Esprinet_20200312", "Name": " Esprinet ", "Date": "2020-03-12 23:14:02", "Volatility ": "5,654", "Value risk": "12,765"}
{"_id": " Eurotech_20200312", "Name": " Eurotech ", "Date": "2020-03-12 23:14:02", "Volatility ": "5,534", "Value risk": "12,832"}

the second Json file is:

{"_id": "Eems_20200312", "Name": "Eems", "Last": "0,066", "%": "-9,04", "Hour": "17:26", "Min": "0,061", "Max": "0,069", "Open": "0,068", "Date": "2020-03-12 23:14:02"}
{"_id": "Esprinet_20200312", "Name": "Esprinet", "Last": "3,495", "%": "-14,86", "Hour": "17:35", "Min": "3,47", "Max": "3,89", "Open": "3,89", "Date": "2020-03-12 23:14:02"}
{"_id": "Eurotech_20200312", "Name": "Eurotech", "Last": "5,05", "%": "-12,17", "Hour": "17:36", "Min": "5,05", "Max": "5,56", "Open": "5,50", "Date": "2020-03-12 23:14:02"}

The result I would like in Mongodb is this:

{"_id": "Eems_20200312", "Name": "Eems", "Last": "0,066", "%": "-9,04", "Hour": "17:26", "Min": "0,061", "Max": "0,069", "Open": "0,068", "Date": "2020-03-12 23:14:02", “Volatility”: “5,328” , “Value Risk”: “12,395”}
{"_id": "Esprinet_20200312", "Name": "Esprinet", "Last": "3,495", "%": "-14,86", "Hour": "17:35", "Min": "3,47", "Max": "3,89", "Open": "3,89", "Date": "2020-03-12 23:14:02", "Volatility ": "5,654", "Value risk": "12,765"}
{"_id": "Eurotech_20200312", "Name": "Eurotech", "Last": "5,05", "%": "-12,17", "Hour": "17:36", "Min": "5,05", "Max": "5,56", "Open": "5,50", "Date": "2020-03-12 23:14:02"", "Volatility ": "5,534", "Value risk": "12,832"}

How can I do it using Nifi?

I would like to merge all the content when the primary key is the same and would like to know if the flow chart is correct or if i need to add something else.

Re: Merge multiple json with the same id in Nifi?

Vj1989 — Sat, 14 Mar 2020 16:38:55 GMT

What you are trying to achieve is can be done using NiFi Lookup Processor. Please explore more on that.

Re: Merge multiple json with the same id in Nifi?

Vj1989 — Sat, 14 Mar 2020 17:51:33 GMT

Refer to this thread for similar question:

https://community.cloudera.com/t5/Support-Questions/merge-too-csv-files-in-nifi/m-p/229259#M191116

question Re: Merge multiple json with the same id in Nifi? in Support Questions

Merge multiple json with the same id in Nifi?

Re: Merge multiple json with the same id in Nifi?

Re: Merge multiple json with the same id in Nifi?