question Re: Jolt transformation Array to Flatten in Support Questions

Jolt transformation Array to Flatten

Althotta — Wed, 26 Apr 2023 15:17:39 GMT

Hi Jolt Experts,

Iam looking to use the Jolt transformation in NIFI and looking for a help.

{
"header": {
"timeStamp": "1681441547144",
"activityId": "5142917973836294867"
},
"resource": {
"drniId": "261968731599587823",
"@type": "Order",
"subtype": "Retail",
"name": "CCCCC",
"resourceCharacteristic": [
{
"propname": "aaa",
"value": "P_1655442"
},
{
"name": "latest",
"value": [
11,
200,
300,
16,
2,
3
]
}]
}
}

To have an output like

{
"records" : [ {
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"drniId" : "261968731599587823",
"type" : "Order",
"subtype" : "Retail",
"name" : "CCCCC",
"propname" : "aaa",
"value" : "P_1655442"

}, {
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"drniId" : "261968731599587823",
"type" : "Order",
"subtype" : "Retail",
"name" : "CCCCC",
"propname" : "latest",
"value" : "11,200,300,16,2,3"
} ]
}

Basically copying all the top header and resource into to new by flattening it. Any help appreciated.

Re: Jolt transformation Array to Flatten

SAMSAL — Wed, 26 Apr 2023 17:58:29 GMT

Hi,

Try the following spec:

[ { // This is to remove the @ character from @type property because I could not // figure out how to make it work in the second shift using escape character '\' "operation": "shift", "spec": { "header": "&", "resource": { "\\@type": "resource.type", "*": "resource.&" } } }, { "operation": "shift", "spec": { "resource": { "resourceCharacteristic": { "*": { "name": "records[&1].propname", "*": "records[&1].&", "@(2,name)": "records[&1].name", "@(2,type)": "records[&1].type", "@(2,subtype)": "records[&1].subtype", "@(2,drniId)": "records[&1].drniId", "@(3,header.activityId)": "records[&1].activityId", "@(3,header.timeStamp)": "records[&1].timeStamp" } } } } }, { "operation": "modify-overwrite-beta", "spec": { "records": { "*": { "value": "=join(',', @(1,value))" } } } } ]

If that helps please accept solution.

Thanks

Re: Jolt transformation Array to Flatten

Althotta — Mon, 01 May 2023 03:10:24 GMT

Thanks @SAMSAL .It worked. In the same way how do i get an simillar output for "resourceRelationshipCharacteristic" . i tried few option but not getting the desirable output.

{
"header": {
"version": "1.0",
"timeStamp": "1681441547144",
"activityId": "5142917973836294867",
"action": "modify"
},
"resource": {
"drniId": "261968731599587823",
"@type": "VVVVV",
"subtype": "BBBBBB",
"name": "VVVVVVV",
"resourceCharacteristic": [
{
"name": "matchingkey",
"value": "CCCCC",
"@type": "string"
}
],
"resourceRelationship": [
{
"drniId": "261968731599587635",
"@type": "CCCCCC",
"subtype": "OTU4",
"name": "BBBBBB",
"relationDrniId": "5143209344417655493",
"resourceRelationshipCharacteristic": [
{
"name": "p3Latest",
"value": true,
"@type": "boolean"
},
{
"name": "isNotInRoute",
"value": "N",
"@type": "string"
}
]
},
{
"drniId": "261988866406271344",
"@type": "BVDFDFS",
"subtype": "VDFDG",
"name": "ABCC",
"relationDrniId": "5143209344417655494",
"resourceRelationshipCharacteristic": [
{
"name": "p3Latest",
"value": true,
"@type": "boolean"
},
{
"name": "isNotInRoute",
"value": "N",
"@type": "string"
}
]
}
]
}
}

Re: Jolt transformation Array to Flatten

SAMSAL — Mon, 01 May 2023 13:30:31 GMT

Hi ,

Can you provide the expected json output? The json you provided seems complex and nested. I'm not sure what exactly you are expecting.

Re: Jolt transformation Array to Flatten

Althotta — Tue, 02 May 2023 00:08:24 GMT

Basically inhertied properties of header and resource inside resourceRelationshipCharacteristic

{
"records" : [ {
"relationDrniId" : "5143209344417655493",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "p3Latest",
"value": true,
"@type": "boolean"

},

{
"relationDrniId" : "5143209344417655493",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "isNotInRoute",
"value": "N",
"@type": "string"

} ,

{
"relationDrniId" : "5143209344417655494",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "p3Latest",
"value": true,
"@type": "boolean"

} ,

"relationDrniId" : "5143209344417655494",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "isNotInRoute",
"value": "N",
"@type": "string"

}

]
}

Re: Jolt transformation Array to Flatten

SAMSAL — Tue, 02 May 2023 16:48:43 GMT

Hi,

This one was a little trickier from the first post, but it seems that there is nothing that you cant do with Jolt 🙂 . Please try the following spec:

[ { // combine all resourceRelationshipCharacteristic under one group // and assign each element under the group unique key depending on // its index location starting from first array under resourceRelationship (&3) and // and ending with nested array resourceRelationshipCharacteristic ($1) so // each element will have unique name 00,01,10,11... "operation": "shift", "spec": { "resource": { "resourceRelationship": { "*": { "resourceRelationshipCharacteristic": { "*": { "@(6,header.action)": "&3&1.action", "@(6,header.timeStamp)": "&3&1.timeStamp", "@(2,relationDrniId)": "&3&1.relationDrniId", "*": "&3&1.&" } } } } } } }, { // bucket each element (00,01,10,11) value into new Array records "operation": "shift", "spec": { "*": "records.[#1]" } } ]

Hope that helps.

I wonder if there is better\cleaner way @araujo @cotopaul @steven-matison

Re: Jolt transformation Array to Flatten

Althotta — Wed, 03 May 2023 02:11:45 GMT

Thanks @SAMSAL You are a genius. I was caught with the duplicate index array and couldn't resolve.

Re: Jolt transformation Array to Flatten

Althotta — Mon, 08 May 2023 12:29:59 GMT

I have used the below spec for moving some of the resource characterstic to top level but iam getting null values.

[
{
"operation": "shift",
"spec": {
"header": {
"timeStamp": "records.inv_activity_ts",
"activityId": "records.inv_activity_id",
"action": "records.action"
},
"resource": {
"drniId": "records.inv_id",
"subtype": "records.inv_subtype",
"name": "records.inv_name",
"resourceCharacteristic": {
"*": {
"name": {
"matchingkey": "records.matchingkey-value",
"status": "records.status-value"
},
"value": {
"matchingkey": "records.matchingkey-value",
"status": "records.status-value"
}
}
}
}
}
}
]

Output iam getting

{
"records" : {
"inv_activity_ts" : "1670484663189",
"inv_activity_id" : "256388257993155783",
"action" : "create",
"inv_id" : "256383859946641699",
"inv_subtype" : "Backplane Connection",
"inv_name" : "Backplane Connection",
"matchingkey-value" : null,
"status-value" : null
}
}

What is mistake iam making?.