Skip to main content
Althotta
Althotta
Silver ⭐️⭐️⭐️
April 26, 2023
Solved

Jolt transformation Array to Flatten

Hi Jolt Experts,

Iam looking to use the Jolt transformation in NIFI and looking for a help.

{
"header": {
"timeStamp": "1681441547144",
"activityId": "5142917973836294867"
},
"resource": {
"drniId": "261968731599587823",
"@type": "Order",
"subtype": "Retail",
"name": "CCCCC",
"resourceCharacteristic": [
{
"propname": "aaa",
"value": "P_1655442"
},
{
"name": "latest",
"value": [
11,
200,
300,
16,
2,
3
]
}]
}
}

 

To have an output like

{
"records" : [ {
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"drniId" : "261968731599587823",
"type" : "Order",
"subtype" : "Retail",
"name" : "CCCCC",
"propname" : "aaa",
"value" : "P_1655442"

}, {
"timeStamp" : "1681441547144",
"activityId" : "5142917973836294867",
"drniId" : "261968731599587823",
"type" : "Order",
"subtype" : "Retail",
"name" : "CCCCC",
"propname" : "latest",
"value" : "11,200,300,16,2,3"
} ]
}

 

Basically copying all the top header and resource into to new by flattening it. Any help appreciated.

 

 

SAMSAL
SAMSALAnswer
Platinum ⭐️⭐️⭐️
May 2, 2023

Hi,

This one was a little trickier from the first post, but it seems that there is nothing that you cant do with Jolt 🙂 . Please try the following spec:

 

[
  {
    // combine all resourceRelationshipCharacteristic under one group
    // and assign each element under the group unique key depending on
    // its index location starting from first array under resourceRelationship (&3) and
    // and ending with nested array resourceRelationshipCharacteristic ($1) so
    // each element will have unique name 00,01,10,11...
    "operation": "shift",
    "spec": {
      "resource": {
        "resourceRelationship": {
          "*": {
            "resourceRelationshipCharacteristic": {
              "*": {
                "@(6,header.action)": "&3&1.action",
                "@(6,header.timeStamp)": "&3&1.timeStamp",
                "@(2,relationDrniId)": "&3&1.relationDrniId",
                "*": "&3&1.&"
              }
            }
          }
        }
      }
    }
  },
  {
    // bucket each element (00,01,10,11) value into new Array records
    "operation": "shift",
    "spec": {
      "*": "records.[#1]"
    }
  }
]

 

Hope that helps.

I wonder if there is better\cleaner way @araujo  @cotopaul @steven-matison 

View in thread

7 replies

SAMSAL
SAMSAL
Platinum ⭐️⭐️⭐️
April 26, 2023

Hi,

Try the following spec:

 

[
  {
    // This is to remove the @ character from @type property because I could not
    // figure out how to make it work in the second shift using escape character '\'
    "operation": "shift",
    "spec": {
      "header": "&",
      "resource": {
        "\\@type": "resource.type",
        "*": "resource.&"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "resource": {
        "resourceCharacteristic": {
          "*": {
            "name": "records[&1].propname",
            "*": "records[&1].&",
            "@(2,name)": "records[&1].name",
            "@(2,type)": "records[&1].type",
            "@(2,subtype)": "records[&1].subtype",
            "@(2,drniId)": "records[&1].drniId",
            "@(3,header.activityId)": "records[&1].activityId",
            "@(3,header.timeStamp)": "records[&1].timeStamp"
          }
        }
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "records": {
        "*": {
          "value": "=join(',', @(1,value))"
        }
      }
    }
  }
]

 

If that helps please accept solution.

Thanks

 

Translate
    Althotta
    AlthottaAuthor
    Silver ⭐️⭐️⭐️
    May 1, 2023

    Thanks @SAMSAL .It worked. In the same way how do i get an simillar output for "resourceRelationshipCharacteristic" . i tried few option but not getting the desirable output.

     

    {
    "header": {
    "version": "1.0",
    "timeStamp": "1681441547144",
    "activityId": "5142917973836294867",
    "action": "modify"
    },
    "resource": {
    "drniId": "261968731599587823",
    "@type": "VVVVV",
    "subtype": "BBBBBB",
    "name": "VVVVVVV",
    "resourceCharacteristic": [
    {
    "name": "matchingkey",
    "value": "CCCCC",
    "@type": "string"
    }
    ],
    "resourceRelationship": [
    {
    "drniId": "261968731599587635",
    "@type": "CCCCCC",
    "subtype": "OTU4",
    "name": "BBBBBB",
    "relationDrniId": "5143209344417655493",
    "resourceRelationshipCharacteristic": [
    {
    "name": "p3Latest",
    "value": true,
    "@type": "boolean"
    },
    {
    "name": "isNotInRoute",
    "value": "N",
    "@type": "string"
    }
    ]
    },
    {
    "drniId": "261988866406271344",
    "@type": "BVDFDFS",
    "subtype": "VDFDG",
    "name": "ABCC",
    "relationDrniId": "5143209344417655494",
    "resourceRelationshipCharacteristic": [
    {
    "name": "p3Latest",
    "value": true,
    "@type": "boolean"
    },
    {
    "name": "isNotInRoute",
    "value": "N",
    "@type": "string"
    }
    ]
    }
    ]
    }
    }

    Translate
    SAMSAL
    SAMSAL
    Platinum ⭐️⭐️⭐️
    May 1, 2023

    Hi ,

    Can you provide the expected json output? The json you provided seems complex and nested. I'm not sure what exactly you are expecting.

    Translate
    Althotta
    AlthottaAuthor
    Silver ⭐️⭐️⭐️
    May 2, 2023

    Basically inhertied properties of header and resource inside resourceRelationshipCharacteristic

     

    {
    "records" : [ {
    "relationDrniId" : "5143209344417655493",
    "timeStamp" : "1681441547144",
    "activityId" : "5142917973836294867",
    "action" : "modify",
    "name": "p3Latest",
    "value": true,
    "@type": "boolean"

    },

    {
    "relationDrniId" : "5143209344417655493",
    "timeStamp" : "1681441547144",
    "activityId" : "5142917973836294867",
    "action" : "modify",
    "name": "isNotInRoute",
    "value": "N",
    "@type": "string"

    } ,




    {
    "relationDrniId" : "5143209344417655494",
    "timeStamp" : "1681441547144",
    "activityId" : "5142917973836294867",
    "action" : "modify",
    "name": "p3Latest",
    "value": true,
    "@type": "boolean"

    } ,


    "relationDrniId" : "5143209344417655494",
    "timeStamp" : "1681441547144",
    "activityId" : "5142917973836294867",
    "action" : "modify",
    "name": "isNotInRoute",
    "value": "N",
    "@type": "string"

    }



    ]
    }

    Translate
    SAMSAL
    SAMSALAnswer
    Platinum ⭐️⭐️⭐️
    May 2, 2023

    Hi,

    This one was a little trickier from the first post, but it seems that there is nothing that you cant do with Jolt 🙂 . Please try the following spec:

     

    [
      {
        // combine all resourceRelationshipCharacteristic under one group
        // and assign each element under the group unique key depending on
        // its index location starting from first array under resourceRelationship (&3) and
        // and ending with nested array resourceRelationshipCharacteristic ($1) so
        // each element will have unique name 00,01,10,11...
        "operation": "shift",
        "spec": {
          "resource": {
            "resourceRelationship": {
              "*": {
                "resourceRelationshipCharacteristic": {
                  "*": {
                    "@(6,header.action)": "&3&1.action",
                    "@(6,header.timeStamp)": "&3&1.timeStamp",
                    "@(2,relationDrniId)": "&3&1.relationDrniId",
                    "*": "&3&1.&"
                  }
                }
              }
            }
          }
        }
      },
      {
        // bucket each element (00,01,10,11) value into new Array records
        "operation": "shift",
        "spec": {
          "*": "records.[#1]"
        }
      }
    ]

     

    Hope that helps.

    I wonder if there is better\cleaner way @araujo  @cotopaul @steven-matison 

    Translate
      Althotta
      AlthottaAuthor
      Silver ⭐️⭐️⭐️
      May 3, 2023

      Thanks @SAMSAL You are a genius. I was caught with the duplicate index array and couldn't resolve. 

      Translate
      Althotta
      AlthottaAuthor
      Silver ⭐️⭐️⭐️
      May 8, 2023

      I have used the below spec for moving some of the resource characterstic to top level but iam getting null values. 

      [
      {
      "operation": "shift",
      "spec": {
      "header": {
      "timeStamp": "records.inv_activity_ts",
      "activityId": "records.inv_activity_id",
      "action": "records.action"
      },
      "resource": {
      "drniId": "records.inv_id",
      "subtype": "records.inv_subtype",
      "name": "records.inv_name",
      "resourceCharacteristic": {
      "*": {
      "name": {
      "matchingkey": "records.matchingkey-value",
      "status": "records.status-value"
      },
      "value": {
      "matchingkey": "records.matchingkey-value",
      "status": "records.status-value"
      }
      }
      }
      }
      }
      }
      ]

       

      Output iam getting

      {
      "records" : {
      "inv_activity_ts" : "1670484663189",
      "inv_activity_id" : "256388257993155783",
      "action" : "create",
      "inv_id" : "256383859946641699",
      "inv_subtype" : "Backplane Connection",
      "inv_name" : "Backplane Connection",
      "matchingkey-value" : null,
      "status-value" : null
      }
      }

       

      What is mistake iam making?. 

      Translate
      Terms & ConditionsAccessibility statement