Support Questions

Find answers, ask questions, and share your expertise

Jolt transformation Array to Flatten

avatar
Rising Star

Hi Jolt Experts,

Iam looking to use the Jolt transformation in NIFI and looking for a help.

{
"header": {
"timeStamp": "1681441547144",
"activityId": "5142917973836294867"
},
"resource": {
"drniId": "261968731599587823",
"@type": "Order",
"subtype": "Retail",
"name": "CCCCC",
"resourceCharacteristic": [
{
"propname": "aaa",
"value": "P_1655442"
},
{
"name": "latest",
"value": [
11,
200,
300,
16,
2,
3
]
}]
}
}

 

To have an output like

{
"records" : [ {
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"drniId" : "261968731599587823",
"type" : "Order",
"subtype" : "Retail",
"name" : "CCCCC",
"propname" : "aaa",
"value" : "P_1655442"

}, {
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"drniId" : "261968731599587823",
"type" : "Order",
"subtype" : "Retail",
"name" : "CCCCC",
"propname" : "latest",
"value" : "11,200,300,16,2,3"
} ]
}

 

Basically copying all the top header and resource into to new by flattening it. Any help appreciated.

 

 

2 ACCEPTED SOLUTIONS

avatar
Super Guru

Hi,

Try the following spec:

 

[
  {
    // This is to remove the @ character from @type property because I could not
    // figure out how to make it work in the second shift using escape character '\'
    "operation": "shift",
    "spec": {
      "header": "&",
      "resource": {
        "\\@type": "resource.type",
        "*": "resource.&"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "resource": {
        "resourceCharacteristic": {
          "*": {
            "name": "records[&1].propname",
            "*": "records[&1].&",
            "@(2,name)": "records[&1].name",
            "@(2,type)": "records[&1].type",
            "@(2,subtype)": "records[&1].subtype",
            "@(2,drniId)": "records[&1].drniId",
            "@(3,header.activityId)": "records[&1].activityId",
            "@(3,header.timeStamp)": "records[&1].timeStamp"
          }
        }
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "records": {
        "*": {
          "value": "=join(',', @(1,value))"
        }
      }
    }
  }
]

 

If that helps please accept solution.

Thanks

 

View solution in original post

avatar
Super Guru

Hi,

This one was a little trickier from the first post, but it seems that there is nothing that you cant do with Jolt 🙂 . Please try the following spec:

 

[
  {
    // combine all resourceRelationshipCharacteristic under one group
    // and assign each element under the group unique key depending on
    // its index location starting from first array under resourceRelationship (&3) and
    // and ending with nested array resourceRelationshipCharacteristic ($1) so
    // each element will have unique name 00,01,10,11...
    "operation": "shift",
    "spec": {
      "resource": {
        "resourceRelationship": {
          "*": {
            "resourceRelationshipCharacteristic": {
              "*": {
                "@(6,header.action)": "&3&1.action",
                "@(6,header.timeStamp)": "&3&1.timeStamp",
                "@(2,relationDrniId)": "&3&1.relationDrniId",
                "*": "&3&1.&"
              }
            }
          }
        }
      }
    }
  },
  {
    // bucket each element (00,01,10,11) value into new Array records
    "operation": "shift",
    "spec": {
      "*": "records.[#1]"
    }
  }
]

 

Hope that helps.

I wonder if there is better\cleaner way @araujo  @cotopaul @steven-matison 

View solution in original post

7 REPLIES 7

avatar
Super Guru

Hi,

Try the following spec:

 

[
  {
    // This is to remove the @ character from @type property because I could not
    // figure out how to make it work in the second shift using escape character '\'
    "operation": "shift",
    "spec": {
      "header": "&",
      "resource": {
        "\\@type": "resource.type",
        "*": "resource.&"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "resource": {
        "resourceCharacteristic": {
          "*": {
            "name": "records[&1].propname",
            "*": "records[&1].&",
            "@(2,name)": "records[&1].name",
            "@(2,type)": "records[&1].type",
            "@(2,subtype)": "records[&1].subtype",
            "@(2,drniId)": "records[&1].drniId",
            "@(3,header.activityId)": "records[&1].activityId",
            "@(3,header.timeStamp)": "records[&1].timeStamp"
          }
        }
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "records": {
        "*": {
          "value": "=join(',', @(1,value))"
        }
      }
    }
  }
]

 

If that helps please accept solution.

Thanks

 

avatar
Rising Star

Thanks @SAMSAL .It worked. In the same way how do i get an simillar output for "resourceRelationshipCharacteristic" . i tried few option but not getting the desirable output.

 

{
"header": {
"version": "1.0",
"timeStamp": "1681441547144",
"activityId": "5142917973836294867",
"action": "modify"
},
"resource": {
"drniId": "261968731599587823",
"@type": "VVVVV",
"subtype": "BBBBBB",
"name": "VVVVVVV",
"resourceCharacteristic": [
{
"name": "matchingkey",
"value": "CCCCC",
"@type": "string"
}
],
"resourceRelationship": [
{
"drniId": "261968731599587635",
"@type": "CCCCCC",
"subtype": "OTU4",
"name": "BBBBBB",
"relationDrniId": "5143209344417655493",
"resourceRelationshipCharacteristic": [
{
"name": "p3Latest",
"value": true,
"@type": "boolean"
},
{
"name": "isNotInRoute",
"value": "N",
"@type": "string"
}
]
},
{
"drniId": "261988866406271344",
"@type": "BVDFDFS",
"subtype": "VDFDG",
"name": "ABCC",
"relationDrniId": "5143209344417655494",
"resourceRelationshipCharacteristic": [
{
"name": "p3Latest",
"value": true,
"@type": "boolean"
},
{
"name": "isNotInRoute",
"value": "N",
"@type": "string"
}
]
}
]
}
}

avatar
Super Guru

Hi ,

Can you provide the expected json output? The json you provided seems complex and nested. I'm not sure what exactly you are expecting.

avatar
Rising Star

Basically inhertied properties of header and resource inside resourceRelationshipCharacteristic

 

{
"records" : [ {
"relationDrniId" : "5143209344417655493",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "p3Latest",
"value": true,
"@type": "boolean"

},

{
"relationDrniId" : "5143209344417655493",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "isNotInRoute",
"value": "N",
"@type": "string"

} ,




{
"relationDrniId" : "5143209344417655494",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "p3Latest",
"value": true,
"@type": "boolean"

} ,


"relationDrniId" : "5143209344417655494",
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"action" : "modify",
"name": "isNotInRoute",
"value": "N",
"@type": "string"

}



]
}

avatar
Super Guru

Hi,

This one was a little trickier from the first post, but it seems that there is nothing that you cant do with Jolt 🙂 . Please try the following spec:

 

[
  {
    // combine all resourceRelationshipCharacteristic under one group
    // and assign each element under the group unique key depending on
    // its index location starting from first array under resourceRelationship (&3) and
    // and ending with nested array resourceRelationshipCharacteristic ($1) so
    // each element will have unique name 00,01,10,11...
    "operation": "shift",
    "spec": {
      "resource": {
        "resourceRelationship": {
          "*": {
            "resourceRelationshipCharacteristic": {
              "*": {
                "@(6,header.action)": "&3&1.action",
                "@(6,header.timeStamp)": "&3&1.timeStamp",
                "@(2,relationDrniId)": "&3&1.relationDrniId",
                "*": "&3&1.&"
              }
            }
          }
        }
      }
    }
  },
  {
    // bucket each element (00,01,10,11) value into new Array records
    "operation": "shift",
    "spec": {
      "*": "records.[#1]"
    }
  }
]

 

Hope that helps.

I wonder if there is better\cleaner way @araujo  @cotopaul @steven-matison 

avatar
Rising Star

Thanks @SAMSAL You are a genius. I was caught with the duplicate index array and couldn't resolve. 

avatar
Rising Star

I have used the below spec for moving some of the resource characterstic to top level but iam getting null values. 

[
{
"operation": "shift",
"spec": {
"header": {
"timeStamp": "records.inv_activity_ts",
"activityId": "records.inv_activity_id",
"action": "records.action"
},
"resource": {
"drniId": "records.inv_id",
"subtype": "records.inv_subtype",
"name": "records.inv_name",
"resourceCharacteristic": {
"*": {
"name": {
"matchingkey": "records.matchingkey-value",
"status": "records.status-value"
},
"value": {
"matchingkey": "records.matchingkey-value",
"status": "records.status-value"
}
}
}
}
}
}
]

 

Output iam getting

{
"records" : {
"inv_activity_ts" : "1670484663189",
"inv_activity_id" : "256388257993155783",
"action" : "create",
"inv_id" : "256383859946641699",
"inv_subtype" : "Backplane Connection",
"inv_name" : "Backplane Connection",
"matchingkey-value" : null,
"status-value" : null
}
}

 

What is mistake iam making?.