Compare data within the JSON using NIFI

[#1]

i have a JSON payload which contains latest and old orders. The data should be compared and output New, update, deleted. How can do it do that in NIFI?.

  "latest_orders": [

    { "order_id": "1001", "customer": "Alice", "amount": 250 },

    { "order_id": "1002", "customer": "Bob", "amount": 300 },

    { "order_id": "1004", "customer": "Diana", "amount": 150 }

  ],

  "old_orders": [

    { "order_id": "1001", "customer": "Alice", "amount": 200 },

    { "order_id": "1002", "customer": "Bob", "amount": 300 },

    { "order_id": "1003", "customer": "Charlie", "amount": 400 }

  ]

}

Output:

    { "order_id": "1001", "customer": "Alice", "amount": 250, "Action":"UPDATE" },

    { "order_id": "1002", "customer": "Bob", "amount": 300 ,"Action":"UPDATE"},

    { "order_id": "1004", "customer": "Diana", "amount": 150,"Action":"NEW" },

    { "order_id": "1003", "customer": "Charlie", "amount": 400,"Action":"DELETE" }

[#2]

Hi @steven-matison @MattWho @mburgess Do you have any insights here? Thanks!

[#3 · Accepted solution]

You can use ExecuteGroovyScript with the following script:

def ff = session.get()
if (!ff) return

def obj = new groovy.json.JsonSlurper().parse(ff.read())
def outObj = []

// Find updated records
def old_ids = obj.old_orders.collect {it.order_id}
def latest_ids = obj.latest_orders.collect {it.order_id}
old_ids.intersect(latest_ids).each {order_id ->
def update_order = obj.latest_orders.find {it.order_id == order_id}
update_order.Action = 'UPDATE'
outObj += update_order
}

// Find deleted records
(old_ids - latest_ids).each {order_id ->
def delete_order = obj.old_orders.find {it.order_id == order_id}
delete_order.Action = 'DELETE'
outObj += delete_order
}

// Find new records
(latest_ids - old_ids).each {order_id ->
def new_order = obj.latest_orders.find {it.order_id == order_id}
new_order.Action = 'NEW'
outObj += new_order
}
ff.write('UTF-8', groovy.json.JsonOutput.toJson(outObj))
REL_SUCCESS << ff

[#4]

Hi @Althotta 

Have you been able to resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.