combining the Json data from two different processors in Apache Nifi

[#1]

Hello,

I am working on one of the tasks in Nifi and I am stuck on of the issue, the issue is I am getting the Json data from jolt transforms processor and also I am getting the Json data from input port processor now the issue is I need to combine the data from two processor based on id’s.  How to combined the data from two processors based on common id ? Will you please help me out.

Thanks,

Sathish

[#2]

Hi,

The full picture is not clear on how your flow works, for example where do you get the json that goes through the jolt processor and input port from? Is there a correlation between getting the json from the different sources where one json is generated before the other or the two json sources are independent of each other? If they are independent of each other then how do you know when to merge , for example what if the json from the input port did not come? Can you explain more on how your flow works and if there is a dependency where one json input drives the other or not?

[#3]

Hello,

The flow is using invoke http processor I will fetch the data from api and inother part the data we are pulling from another api sending the data to ouput port, now using input port I am pulling the data and there is common field which is id, using the id from both apis we need to merge the data that  is requirement, I will tell you the flow in diagrammatic way.

[#4 · Accepted solution]

Hi,

Thanks for providing the diagram. Not sure if you can have both InvokeHttp in one group where one comes after the other instead of using two groups and input\output ports, but if you can then you can take advantage of the ForkEnrichment\JoinEnrichment processors (available in 1.16 and higher) using the following pattern:

If both invokehttp output have the same order for the records then you can use the Wrapper or Insert Enrichment strategy, if the records are not in the same order you can use SQL strategy. For more information check the JoinEnrichment user manual :

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.17.0/org.apach...

Hope that helps. If it does please accept solution.

[#5]

Thanks for your help @SAMSAL