- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Creating a value in json only if the key exists using Jolt
- Labels:
-
Apache NiFi
Created ‎11-20-2023 07:43 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am new to Jolt, I am trying to transform JSON to a particular format using nifi, where i am moving some objects into another object, I think i am close
This is my JSON
{
"relatedParty": [
{
"id": "B2BSmallandMedium"
}
],
"serviceOrderItem": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26f4",
"action": "add",
"service": {
"name": "cust",
"state": "active",
"supportingService": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26e99",
"href": "serviceInventory/v4/service"
},
{
"id": "7313c885-a075-4188-a6bc-c3fbfb4c7422",
"href": "activationAndConfiguration/v4/service"
}
]
}
}
]
}
I am new to Jolt, I am trying to transform JSON to a particular format, where i am moving some objects into another object, I think i am close.
This is my JSON
{
"relatedParty": [
{
"id": "B2BSmallandMedium"
}
],
"serviceOrderItem": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26f4",
"action": "add",
"service": {
"name": "cust",
"state": "active",
"supportingService": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26e99",
"href": "serviceInventory/v4/service"
},
{
"id": "7313c885-a075-4188-a6bc-c3fbfb4c7422",
"href": "activationAndConfiguration/v4/service"
}
]
}
}
]
}
I am using the JOLT
[
{
"operation": "shift",
"spec": {
"relatedParty": "relatedParty",
"serviceOrderItem": {
"*": {
"id": "serviceOrderItem[&1].id",
"action": "serviceOrderItem[&1].action",
"service": {
"*": "serviceOrderItem[&2].service.&",
"#modifyPath": "serviceOrderItem[&2].service.serviceCharacteristic[0].name",
"@(1,modifyPath)": "serviceOrderItem[&2].service.serviceCharacteristic[0].value",
"#supportingService": "serviceOrderItem[&2].service.serviceCharacteristic[1].name",
"@(1,service.supportingService)": "serviceOrderItem[&2].service.serviceCharacteristic[1].value"
}
}
}
}
},
{
"operation": "modify-default-beta",
"spec": {
"serviceOrderItem": {
"*": {
"service": {
"serviceCharacteristic": {
"*": {
"value": []
}
}
}
}
}
}
},
{
"operation": "remove",
"spec": {
"serviceOrderItem": {
"*": {
"service": {
"supportingService": ""
}
}
}
}
}]
this is now giving me
"service" : {
"name" : "cust",
"serviceCharacteristic" : [ {
"name" : "modifyPath",
"value" : "[]"
}, {
"name" : "supportingService",
"value" : [ {
"href" : "serviceInventory/v4/service",
"id" : "e26205f4-e144-4ded-9e64-82ede0b26e99"
}, {
"href" : "activationAndConfiguration/v4/service",
"id" : "7313c885-a075-4188-a6bc-c3fbfb4c7422"
} ]
} ],
"state" : "active"
}
} ]
This is working for me but,
I would like to know if it is possible replace:
"#modifyPath": "serviceOrderItem[&2].service.serviceCharacteristic[0].name"
"#supportingService": "serviceOrderItem[&2].service.serviceCharacteristic[1].name"
to update the Json only when the key is actually available. I do not want an empty array.
Thank you for your time
Created on ‎12-05-2023 03:49 AM - edited ‎12-05-2023 03:49 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I can see the full problem now. I wish I have considered it in my first response because it will change things a little bit.
Since you have mentioned that you could have other arrays in each level, the current spec basically wont work unless you create different specs for different scenarios which is not a good idea.
This can be resolved in two shift transformations:
1- Bring those array objects from different levels into "serviceCharacteristic" array.
2- Apply the name, value assignment based on each object array index in "serviceCharacteristic" array.
In this case no need to figure out how many arrays objects you might have in the each level, nor you have to assign hard coded index.
Here is the final spec:
[
// First transformation, group array objects under service
// and serviceOrderItem into one array "serviceCharacteristic"
{
"operation": "shift",
"spec": {
"*": "&",
"serviceOrderItem": {
"*": {
"id": "serviceOrderItem.[&1].&",
"action": "serviceOrderItem.[&1].&",
"service": {
"id": "serviceOrderItem.[&2].service.&",
"state": "serviceOrderItem.[&2].service.&",
//group any array objects at the supportingService level into serviceCharacteristic
"*": "serviceOrderItem.[&2].service.serviceCharacteristic[].&"
},
//group any array objects at the modifyPath level into serviceCharacteristic
"*": "serviceOrderItem.[&1].service.serviceCharacteristic[].&"
}
}
}
},
// 2ed transformation to assign name, value for each
// serviceCharacteristic array object
{
"operation": "shift",
"spec": {
"*": "&",
"serviceOrderItem": {
"*": {
"*": "serviceOrderItem.[&1].&",
"service": {
"*": "serviceOrderItem.[&2].service.&",
"serviceCharacteristic": {
"*": {
"*": {
"$": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].name",
"@": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].value"
}
}
}
}
}
}
}
}
]
Note: Not sure why you need the "recursivelySquashNulls" transformation. I did not see any use case for here but you can add it if it solves other issues.
Hope that helps.
Created ‎11-20-2023 09:52 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Changed the jolt to
[
{
"operation": "shift",
"spec": {
"relatedParty": "relatedParty",
"serviceOrderItem": {
"*": {
"id": "serviceOrderItem[&1].id",
"action": "serviceOrderItem[&1].action",
"service": {
"*": "serviceOrderItem[&2].service.&",
"supportingService": {
"#supportingService": "serviceOrderItem[&3].service.serviceCharacteristic[1].name",
"*": "serviceOrderItem[&3].service.serviceCharacteristic[1].value"
}
},
"modifyPath": {
"#modifyPath": "serviceOrderItem[&2].service.serviceCharacteristic[1].name",
"@(1,modifyPath)": "serviceOrderItem[&2].service.serviceCharacteristic[1].value"
}
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
}]
Is there a way to remove the indexing from serviceCharacteristic[1].name / serviceCharacteristic[0].name ?
Created ‎11-28-2023 03:54 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You don't have to put it back into an array [] if you don't want.
Created ‎11-30-2023 03:50 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@joseomjr, Infact i want it in an array, as objects inside serviceCharacteristics array
I was wondering if i could avoid the 1 inside serviceCharacteristics[1] rather put a more generic indexing
Created ‎11-30-2023 07:07 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
A gotcha, you can do what you already do in your JOLT and use & instead of using 1
Created ‎11-30-2023 07:32 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @scoutjohn ,
Your spec can be written as follows:
[
{
"operation": "shift",
"spec": {
"*": "&",
"serviceOrderItem": {
"*": {
"*": "serviceOrderItem.[&1].&",
"service": {
"*": "serviceOrderItem.[&2].service.&",
"supportingService": {
"$": "serviceOrderItem.[&3].service.serviceCharacteristic[#].name",
"@": "serviceOrderItem.[&3].service.serviceCharacteristic[#].value"
}
}
}
}
}
}]
I did not not add the "modifyPath" because I did not see anything related to this object in the provided json input.
Notice how I used # for serviceCharacteristic[#].name & serviceCharacteristic[#].value which tells the spec to group everything under one object under the serviceCharacteristic array.
If you find this helpful please accept solution.
Thanks
Created ‎12-04-2023 09:35 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@SAMSAL , Sorry for the delay,
Thank you for your solution.
{
"relatedParty": [
{
"id": "B2BSmallandMediumBusiness",
"role": "InstanceConsumerGroup",
"@referredType": "InstanceConsumerGroup"
}
],
"serviceOrderItem": [
{
"id": "0965f63b-db26-4397-b222-190a891838f0",
"action": "modify",
"service": {
"id": "cust",
"state": "active",
"supportingService": [
{
"id": "e26205f4-e144-4ded-9e64-82ede0b26e99",
"href": "serviceInventory/v4/service"
}
]
},
"modifyPath": [
{
"op": "add",
"path": "$.supportingService"
}
]
}
]
}
for the incoming request modifyPath will come outside the service object , supportingService array will come inside service object. the output should have serviceCharacteristic array inside service object with name and value ( value being the whole array moved from modifyPath in this in case),
"serviceCharacteristic" : [ {
"name" : "modifyPath",
"value" : "[]"
},
{ "name" : "supportingService",
"value" : "[]"
}
]
like the attached screenshot
The input could change , can also have other arrays that need to be moved into the serviceCharacteristic array, was trying to understand if it is possible to avoid the indexing on serviceCharacteristic in jolt transformation.
Created on ‎12-05-2023 03:49 AM - edited ‎12-05-2023 03:49 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I can see the full problem now. I wish I have considered it in my first response because it will change things a little bit.
Since you have mentioned that you could have other arrays in each level, the current spec basically wont work unless you create different specs for different scenarios which is not a good idea.
This can be resolved in two shift transformations:
1- Bring those array objects from different levels into "serviceCharacteristic" array.
2- Apply the name, value assignment based on each object array index in "serviceCharacteristic" array.
In this case no need to figure out how many arrays objects you might have in the each level, nor you have to assign hard coded index.
Here is the final spec:
[
// First transformation, group array objects under service
// and serviceOrderItem into one array "serviceCharacteristic"
{
"operation": "shift",
"spec": {
"*": "&",
"serviceOrderItem": {
"*": {
"id": "serviceOrderItem.[&1].&",
"action": "serviceOrderItem.[&1].&",
"service": {
"id": "serviceOrderItem.[&2].service.&",
"state": "serviceOrderItem.[&2].service.&",
//group any array objects at the supportingService level into serviceCharacteristic
"*": "serviceOrderItem.[&2].service.serviceCharacteristic[].&"
},
//group any array objects at the modifyPath level into serviceCharacteristic
"*": "serviceOrderItem.[&1].service.serviceCharacteristic[].&"
}
}
}
},
// 2ed transformation to assign name, value for each
// serviceCharacteristic array object
{
"operation": "shift",
"spec": {
"*": "&",
"serviceOrderItem": {
"*": {
"*": "serviceOrderItem.[&1].&",
"service": {
"*": "serviceOrderItem.[&2].service.&",
"serviceCharacteristic": {
"*": {
"*": {
"$": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].name",
"@": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].value"
}
}
}
}
}
}
}
}
]
Note: Not sure why you need the "recursivelySquashNulls" transformation. I did not see any use case for here but you can add it if it solves other issues.
Hope that helps.
Created on ‎12-05-2023 04:41 AM - edited ‎12-05-2023 04:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
