Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What is easiest way to process a flowfile containing a list of JSON objects into a list of documents for mongo

Highlighted

What is easiest way to process a flowfile containing a list of JSON objects into a list of documents for mongo

New Contributor

I have a flowfile that contains a list of JSON objects. For each JSON object in the list, I need to generate a single call to PutMongo to place that object into Mongo. The key will be the evt_data.targetDataId. The call to PutMongo will insert or update the record for that targetDataId.


I'm looking for the most efficient way to process such an array. Do I need to first need to use an intermediate processor (e.g. SplitJSON) to reduce it to a list of flowfiles, one per object in the array, before sending to PutMongo? If so, what processor should I use for that?


For example, if I have a flowfile that looks like the following. This would result in 2 inserts (or updates) using PutMongo. It would be really nice if PutMongo supported this conversion of an array of documents, to an array of "inserted/updated" documents, out of the box.

[
{
"evt_type":"wheels_down",
"callsign":"C887",
"evt_data":{
"targetDataId":633901919919,
"sourceName":"STDDS",
"callsign":"C887",
"sourceSpecificGufi":[
"KORD_C887_2715",
"KORD_5868622"
],
"updateTime":1563478106000,
"targetType":"VEHICLE",
"subSource":"KORD"
}
},
{
"evt_type":"wheels_up",
"callsign":"C888",
"evt_data":{
"targetDataId":633901919919,
"sourceName":"STDDS",
"callsign":"C888",
"sourceSpecificGufi":[
"KORD_C888_2715",
"KORD_5868622"
],
"updateTime":1563478106000,
"targetType":"VEHICLE",
"subSource":"KORD"
}
}
]