Support Questions

Find answers, ask questions, and share your expertise

Compare data within the JSON using NIFI

avatar
Rising Star

i have a JSON payload which contains latest and old orders. The data should be compared and output New, update, deleted. How can do it do that in NIFI?.

  "latest_orders": [
    { "order_id": "1001", "customer": "Alice", "amount": 250 },
    { "order_id": "1002", "customer": "Bob", "amount": 300 },
    { "order_id": "1004", "customer": "Diana", "amount": 150 }
  ],
  "old_orders": [
    { "order_id": "1001", "customer": "Alice", "amount": 200 },
    { "order_id": "1002", "customer": "Bob", "amount": 300 },
    { "order_id": "1003", "customer": "Charlie", "amount": 400 }
  ]
}
Output:
    { "order_id": "1001", "customer": "Alice", "amount": 250, "Action":"UPDATE" },
    { "order_id": "1002", "customer": "Bob", "amount": 300 ,"Action":"UPDATE"},
    { "order_id": "1004", "customer": "Diana", "amount": 150,"Action":"NEW" },
    { "order_id": "1003", "customer": "Charlie", "amount": 400,"Action":"DELETE" }
 
2 REPLIES 2

avatar
Community Manager

Hi @steven-matison @MattWho @mburgess Do you have any insights here? Thanks!


Regards,

Diana Torres,
Senior Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Master Guru

You can use ExecuteGroovyScript with the following script:

def ff = session.get()
if (!ff) return

def obj = new groovy.json.JsonSlurper().parse(ff.read())
def outObj = []

// Find updated records
def old_ids = obj.old_orders.collect {it.order_id}
def latest_ids = obj.latest_orders.collect {it.order_id}
old_ids.intersect(latest_ids).each {order_id ->
def update_order = obj.latest_orders.find {it.order_id == order_id}
update_order.Action = 'UPDATE'
outObj += update_order
}

// Find deleted records
(old_ids - latest_ids).each {order_id ->
def delete_order = obj.old_orders.find {it.order_id == order_id}
delete_order.Action = 'DELETE'
outObj += delete_order
}

// Find new records
(latest_ids - old_ids).each {order_id ->
def new_order = obj.latest_orders.find {it.order_id == order_id}
new_order.Action = 'NEW'
outObj += new_order
}
ff.write('UTF-8', groovy.json.JsonOutput.toJson(outObj))
REL_SUCCESS << ff