Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi - Which processor should I use to combine different JSON inputs into one object?

Nifi - Which processor should I use to combine different JSON inputs into one object?

New Contributor

85498-capture.jpg

I'd like to set up a dataflow that takes in multiple JSON inputs, and combines them into a single JSON object with multiple properties (I'm currently using a few GenerateFlowFile processors to generate the inputs), and send the data every 10 seconds via the PublishMQTT processor.

The inputs come in at different intervals (1-5seconds), and examples are:

- {"temperature": 60}

- {"pressure": 30}

I would like to compile the incoming data into one object i.e. {"temperature": 60,"pressure": 30} before sending it to the PublishMQTT processor.

Also, if fresh data with the same attribute comes in before the message is sent, it should update the attribute in the same object instead of being queued. i.e. If new data entered {"pressure": 150}, the output object should be updated to {"temperature": 60,"pressure": 150} before it is sent out via MQTT

I'm guessing that I will require a processor (see blue circle in attached image), but I'm not sure what processor(s) does what I've described.

Thanks in advance!

2 REPLIES 2
Highlighted

Re: Nifi - Which processor should I use to combine different JSON inputs into one object?

Master Collaborator

@Edwin Chua The processors you are looking for is MergeContent and MergeRecord:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache...

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Highlighted

Re: Nifi - Which processor should I use to combine different JSON inputs into one object?

Explorer

As for the second question you may need processors operating with map cache to compare previous cached value of pressure with the recent one

Don't have an account?
Coming from Hortonworks? Activate your account here