Created on 03-14-2020 09:16 AM - last edited on 03-14-2020 10:00 AM by ask_bill_brooks
I have this flowchart in nifi, where json files (ConsumerKafka) are consumed and then pass them in mongodb
the first Json file is:
{"_id": "Eems_20200312", "Name": "Eems", "Date": "2020-03-12 23:14:02", "Volatility ": "5,328", "Value risk": "12,395"}
{"_id": " Esprinet_20200312", "Name": " Esprinet ", "Date": "2020-03-12 23:14:02", "Volatility ": "5,654", "Value risk": "12,765"}
{"_id": " Eurotech_20200312", "Name": " Eurotech ", "Date": "2020-03-12 23:14:02", "Volatility ": "5,534", "Value risk": "12,832"}
the second Json file is:
{"_id": "Eems_20200312", "Name": "Eems", "Last": "0,066", "%": "-9,04", "Hour": "17:26", "Min": "0,061", "Max": "0,069", "Open": "0,068", "Date": "2020-03-12 23:14:02"}
{"_id": "Esprinet_20200312", "Name": "Esprinet", "Last": "3,495", "%": "-14,86", "Hour": "17:35", "Min": "3,47", "Max": "3,89", "Open": "3,89", "Date": "2020-03-12 23:14:02"}
{"_id": "Eurotech_20200312", "Name": "Eurotech", "Last": "5,05", "%": "-12,17", "Hour": "17:36", "Min": "5,05", "Max": "5,56", "Open": "5,50", "Date": "2020-03-12 23:14:02"}
The result I would like in Mongodb is this:
{"_id": "Eems_20200312", "Name": "Eems", "Last": "0,066", "%": "-9,04", "Hour": "17:26", "Min": "0,061", "Max": "0,069", "Open": "0,068", "Date": "2020-03-12 23:14:02", “Volatility”: “5,328” , “Value Risk”: “12,395”}
{"_id": "Esprinet_20200312", "Name": "Esprinet", "Last": "3,495", "%": "-14,86", "Hour": "17:35", "Min": "3,47", "Max": "3,89", "Open": "3,89", "Date": "2020-03-12 23:14:02", "Volatility ": "5,654", "Value risk": "12,765"}
{"_id": "Eurotech_20200312", "Name": "Eurotech", "Last": "5,05", "%": "-12,17", "Hour": "17:36", "Min": "5,05", "Max": "5,56", "Open": "5,50", "Date": "2020-03-12 23:14:02"", "Volatility ": "5,534", "Value risk": "12,832"}
How can I do it using Nifi?
I would like to merge all the content when the primary key is the same and would like to know if the flow chart is correct or if i need to add something else.
Created 03-14-2020 10:51 AM
Refer to this thread for similar question:
https://community.cloudera.com/t5/Support-Questions/merge-too-csv-files-in-nifi/m-p/229259#M191116
Created 03-14-2020 09:38 AM
What you are trying to achieve is can be done using NiFi Lookup Processor. Please explore more on that.
Created 03-14-2020 10:51 AM
Refer to this thread for similar question:
https://community.cloudera.com/t5/Support-Questions/merge-too-csv-files-in-nifi/m-p/229259#M191116