Skip to main content
scoutjohn
scoutjohn
Gold ⭐️
November 21, 2023
Solved

Creating a value in json only if the key exists using Jolt

I am new to Jolt, I am trying to transform JSON to a particular format using nifi, where i am moving some objects into another object, I think i am close

This is my JSON

{
"relatedParty": [
{
"id": "B2BSmallandMedium"
}
],
"serviceOrderItem": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26f4",
"action": "add",
"service": {
"name": "cust",
"state": "active",
"supportingService": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26e99",
"href": "serviceInventory/v4/service"
},
{
"id": "7313c885-a075-4188-a6bc-c3fbfb4c7422",
"href": "activationAndConfiguration/v4/service"
}
]
}
}
]
}

I am new to Jolt, I am trying to transform JSON to a particular format, where i am moving some objects into another object, I think i am close.

This is my JSON

{
"relatedParty": [
{
"id": "B2BSmallandMedium"
}
],
"serviceOrderItem": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26f4",
"action": "add",
"service": {
"name": "cust",
"state": "active",
"supportingService": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26e99",
"href": "serviceInventory/v4/service"
},
{
"id": "7313c885-a075-4188-a6bc-c3fbfb4c7422",
"href": "activationAndConfiguration/v4/service"
}
]
}
}
]
}

I am using the JOLT

 

 

[
  {
    "operation": "shift",
    "spec": {
      "relatedParty": "relatedParty",
      "serviceOrderItem": {
        "*": {
          "id": "serviceOrderItem[&1].id",
          "action": "serviceOrderItem[&1].action",
          "service": {
            "*": "serviceOrderItem[&2].service.&",
            "#modifyPath": "serviceOrderItem[&2].service.serviceCharacteristic[0].name",
            "@(1,modifyPath)": "serviceOrderItem[&2].service.serviceCharacteristic[0].value",
            "#supportingService": "serviceOrderItem[&2].service.serviceCharacteristic[1].name",
            "@(1,service.supportingService)": "serviceOrderItem[&2].service.serviceCharacteristic[1].value"
          }
        }
      }
    }
},
  {
    "operation": "modify-default-beta",
    "spec": {
      "serviceOrderItem": {
        "*": {
          "service": {
            "serviceCharacteristic": {
              "*": {
                "value": []
              }
            }
          }
        }
      }
    }
},
  {
    "operation": "remove",
    "spec": {
      "serviceOrderItem": {
        "*": {
          "service": {
            "supportingService": ""
          }
        }
      }
    }
}]

 

 

this is now giving me

"service" : {
"name" : "cust",
"serviceCharacteristic" : [ {
"name" : "modifyPath",
"value" : "[]"
}, {
"name" : "supportingService",
"value" : [ {
"href" : "serviceInventory/v4/service",
"id" : "e26205f4-e144-4ded-9e64-82ede0b26e99"
}, {
"href" : "activationAndConfiguration/v4/service",
"id" : "7313c885-a075-4188-a6bc-c3fbfb4c7422"
} ]
} ],
"state" : "active"
}
} ]

This is working for me but,

I would like to know if it is possible replace:

"#modifyPath": "serviceOrderItem[&2].service.serviceCharacteristic[0].name"

"#supportingService": "serviceOrderItem[&2].service.serviceCharacteristic[1].name"

to update the Json only when the key is actually available. I do not want an empty array.

Thank you for your time

 

SAMSAL
SAMSALAnswer
Platinum ⭐️⭐️⭐️
December 5, 2023

@scoutjohn,

I can see the full problem now. I wish I have considered it in my first response because it will change things a little bit.

Since you have mentioned that you could have other arrays in each level, the current spec basically wont work unless you create different specs for different scenarios which is not a good idea.

This can be resolved in two shift transformations:

1- Bring those array objects from different levels  into "serviceCharacteristic" array.

2- Apply the name, value assignment based on each object array index in "serviceCharacteristic" array.

In this case no need to figure out how many arrays objects you might have in the each level, nor you have to assign hard coded index.

Here is the final spec:

 

[
  // First transformation, group array objects under service
  // and serviceOrderItem into one array "serviceCharacteristic"
  {
    "operation": "shift",
    "spec": {
      "*": "&",
      "serviceOrderItem": {
        "*": {
          "id": "serviceOrderItem.[&1].&",
          "action": "serviceOrderItem.[&1].&",
          "service": {
            "id": "serviceOrderItem.[&2].service.&",
            "state": "serviceOrderItem.[&2].service.&",
            //group any array objects at the supportingService level into serviceCharacteristic
            "*": "serviceOrderItem.[&2].service.serviceCharacteristic[].&"
          },
          //group any array objects at the modifyPath level into serviceCharacteristic
          "*": "serviceOrderItem.[&1].service.serviceCharacteristic[].&"
        }
      }
    }
  },
  // 2ed transformation to assign name, value for each
  // serviceCharacteristic array object
  {
    "operation": "shift",
    "spec": {
      "*": "&",
      "serviceOrderItem": {
        "*": {
          "*": "serviceOrderItem.[&1].&",
          "service": {
            "*": "serviceOrderItem.[&2].service.&",
            "serviceCharacteristic": {
              "*": {
                "*": {
                  "$": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].name",
                  "@": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].value"
                }
              }
            }
          }
        }
      }
    }
  }
]

 

Note: Not sure why you need the "recursivelySquashNulls"  transformation. I did not see any use case for here but you can add it if it solves other issues.

Hope that helps.

 

View in thread

8 replies

scoutjohn
scoutjohnAuthor
Gold ⭐️
November 21, 2023

Changed the jolt to 

[
  {
    "operation": "shift",
    "spec": {
      "relatedParty": "relatedParty",
      "serviceOrderItem": {
        "*": {
          "id": "serviceOrderItem[&1].id",
          "action": "serviceOrderItem[&1].action",
          "service": {
            "*": "serviceOrderItem[&2].service.&",
            "supportingService": {
              "#supportingService": "serviceOrderItem[&3].service.serviceCharacteristic[1].name",
              "*": "serviceOrderItem[&3].service.serviceCharacteristic[1].value"
            }
          },
          "modifyPath": {
            "#modifyPath": "serviceOrderItem[&2].service.serviceCharacteristic[1].name",
            "@(1,modifyPath)": "serviceOrderItem[&2].service.serviceCharacteristic[1].value"
          }
        }
      }
    }
},
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": "=recursivelySquashNulls"
    }
  }]

Is there a way to remove the indexing from serviceCharacteristic[1].name / serviceCharacteristic[0].name ?

 

Translate
joseomjr
joseomjr
Gold ⭐️⭐️
November 28, 2023

You don't have to put it back into an array [] if you don't want.

Translate
scoutjohn
scoutjohnAuthor
Gold ⭐️
November 30, 2023

@joseomjr,  Infact i want it in an array, as objects inside serviceCharacteristics array 

I was wondering if i could avoid the 1 inside serviceCharacteristics[1] rather put a more generic indexing

Translate
joseomjr
joseomjr
Gold ⭐️⭐️
November 30, 2023

A gotcha, you can do what you already do in your JOLT and use & instead of using 1

Translate
SAMSAL
SAMSAL
Platinum ⭐️⭐️⭐️
November 30, 2023

Hi @scoutjohn ,

Your spec can be written as follows:

 

[
  {
    "operation": "shift",
    "spec": {
      "*": "&",
      "serviceOrderItem": {
        "*": {
          "*": "serviceOrderItem.[&1].&",
          "service": {
            "*": "serviceOrderItem.[&2].service.&",
            "supportingService": {
              "$": "serviceOrderItem.[&3].service.serviceCharacteristic[#].name",
              "@": "serviceOrderItem.[&3].service.serviceCharacteristic[#].value"
            }
          }
        }
      }
    }
  }]

I did not not add the  "modifyPath" because I did not see anything related to this object in the provided json input.

Notice how I used # for serviceCharacteristic[#].name & serviceCharacteristic[#].value which tells the spec to group everything under one object under the serviceCharacteristic array.

If you find this helpful please accept solution.

Thanks

Translate
scoutjohn
scoutjohnAuthor
Gold ⭐️
December 5, 2023

@SAMSAL , Sorry for the delay, 
Thank you for your solution. 

{
	"relatedParty": [
		{
			"id": "B2BSmallandMediumBusiness",
			"role": "InstanceConsumerGroup",
			"@referredType": "InstanceConsumerGroup"
		}
	],
	"serviceOrderItem": [
		{
			"id": "0965f63b-db26-4397-b222-190a891838f0",
			"action": "modify",
			"service": {
				"id": "cust",
				"state": "active",
				"supportingService": [
					{
						"id": "e26205f4-e144-4ded-9e64-82ede0b26e99",
						"href": "serviceInventory/v4/service"
					}
					]
			},
			"modifyPath": [
				{
					"op": "add",
					"path": "$.supportingService"
				}
			]
		}
	]
}

for the incoming request modifyPath will come outside the service object , supportingService array will come inside service object. the output should have serviceCharacteristic array inside service object with name and value ( value being the whole array moved from modifyPath in this in case), 

"serviceCharacteristic" : [ {
"name" : "modifyPath",
"value" : "[]"
},
{ "name" : "supportingService",
"value" : "[]"
}
]

like the attached screenshot

scoutjohn_0-1701754506232.png

 


The input could change , can also have other arrays that need to be moved into the serviceCharacteristic array, was trying to understand if it is possible to avoid the indexing on serviceCharacteristic in jolt transformation.





Translate
SAMSAL
SAMSALAnswer
Platinum ⭐️⭐️⭐️
December 5, 2023

@scoutjohn,

I can see the full problem now. I wish I have considered it in my first response because it will change things a little bit.

Since you have mentioned that you could have other arrays in each level, the current spec basically wont work unless you create different specs for different scenarios which is not a good idea.

This can be resolved in two shift transformations:

1- Bring those array objects from different levels  into "serviceCharacteristic" array.

2- Apply the name, value assignment based on each object array index in "serviceCharacteristic" array.

In this case no need to figure out how many arrays objects you might have in the each level, nor you have to assign hard coded index.

Here is the final spec:

 

[
  // First transformation, group array objects under service
  // and serviceOrderItem into one array "serviceCharacteristic"
  {
    "operation": "shift",
    "spec": {
      "*": "&",
      "serviceOrderItem": {
        "*": {
          "id": "serviceOrderItem.[&1].&",
          "action": "serviceOrderItem.[&1].&",
          "service": {
            "id": "serviceOrderItem.[&2].service.&",
            "state": "serviceOrderItem.[&2].service.&",
            //group any array objects at the supportingService level into serviceCharacteristic
            "*": "serviceOrderItem.[&2].service.serviceCharacteristic[].&"
          },
          //group any array objects at the modifyPath level into serviceCharacteristic
          "*": "serviceOrderItem.[&1].service.serviceCharacteristic[].&"
        }
      }
    }
  },
  // 2ed transformation to assign name, value for each
  // serviceCharacteristic array object
  {
    "operation": "shift",
    "spec": {
      "*": "&",
      "serviceOrderItem": {
        "*": {
          "*": "serviceOrderItem.[&1].&",
          "service": {
            "*": "serviceOrderItem.[&2].service.&",
            "serviceCharacteristic": {
              "*": {
                "*": {
                  "$": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].name",
                  "@": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].value"
                }
              }
            }
          }
        }
      }
    }
  }
]

 

Note: Not sure why you need the "recursivelySquashNulls"  transformation. I did not see any use case for here but you can add it if it solves other issues.

Hope that helps.

 

Translate
scoutjohn
scoutjohnAuthor
Gold ⭐️
December 5, 2023

@SAMSAL , Thank you. this works.

scoutjohn_0-1701780207080.png

 

Translate
Terms & ConditionsAccessibility statement