question JOLT Nested Array in Support Questions

JOLT Nested Array

Syed0000 — Tue, 29 Oct 2024 21:38:58 GMT

Hi,

Could be please help me why categoryList is combining all values inside one list instead of 2nd index of paths object , like productList is working fine only difference is that i added one more condition for categoryList that is causing the issue. could you please suggest what i am missing here ?

Input

{ "correlationId": "4c7f8c69-36eb-4f65-bded-6ed08503631a", "sourceSystemId": "PCS", "messageTimestamp": "2024-10-29T13:10:54.297-04:00", "id": 2173, "promotionId": 569, "offerId": 2173, "description": "Price Rule wIncl and Excl - HCO UK", "levelCode": 0, "typeCode": 0, "templateId": 4, "isCouponCodeRequired": 0, "startDateTime": "2024-10-30T00:00:00.000Z", "endDateTime": "2024-10-31T23:59:00.000Z", "exclusiveDiscountInd": 0, "tiers": [ { "conditions": [ { "conditionId": 374, "buySpendType": 0, "buySpendValue": 1, "buyUom": "EA", "conditionsMerch": [ { "merchandiseLevel": 5, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700348", "excludeInd": 0, "keyItemNum": "KIC_123444", "conditionMerchId": 3110 }, { "merchandiseLevel": 5, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700348", "excludeInd": 1, "keyItemNum": "KIC_1111111", "conditionMerchId": 3110 }, { "merchandiseLevel": 6, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700348", "excludeInd": 0, "webCategoryId": "WEBCTGY_85265", "conditionMerchId": 3110 }, { "merchandiseLevel": 6, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700364", "excludeInd": 1, "webCategoryId": "WEBCTGY_166330", "conditionMerchId": 3111 } ] } ], "rewards": [ { "rewardId": 2293, "changeType": 2, "changeAmount": 30, "changePercent": 0, "quantityToDiscount": 99, "applyInd": 2, "rewardsMerch": [] } ], "priceRuleAttributes": [ { "id": "PR_EMPTY_PATH_NAME", "type": "CFA", "values": [ "No Everbody Pricing" ] }, { "id": "PR_PATH_NAME", "type": "CFA", "values": [ "Members Save on HCO AE" ] }, { "id": "PR_TARGET_SEGMENT", "type": "CFA", "values": [ "LOGIN" ] } ] }, { "conditions": [ { "conditionId": 371, "buySpendType": 0, "buySpendValue": 1, "buyUom": "EA", "conditionsMerch": [ { "merchandiseLevel": 5, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700348", "excludeInd": 1, "keyItemNum": "KIC_1111111", "conditionMerchId": 3110 }, { "merchandiseLevel": 5, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700348", "excludeInd": 0, "keyItemNum": "KIC_123444", "conditionMerchId": 3110 }, { "merchandiseLevel": 6, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700348", "excludeInd": 1, "webCategoryId": "WEBCTGY_85265", "conditionMerchId": 3110 }, { "merchandiseLevel": 6, "dept": 0, "deptClass": 0, "uniqueClass": 0, "subClass": 0, "uniqueSubClass": 0, "item": "101700364", "excludeInd": 0, "webCategoryId": "WEBCTGY_166330", "conditionMerchId": 3105 } ] } ], "rewards": [ { "rewardId": 2290, "changeType": 2, "changeAmount": 20, "changePercent": 0, "quantityToDiscount": 99, "applyInd": 2, "rewardsMerch": [] } ], "priceRuleAttributes": [ { "id": "PR_PATH_NAME", "type": "CFA", "values": [ "Members Save on Bottoms" ] }, { "id": "PR_TARGET_SEGMENT", "type": "CFA", "values": [ "LOGIN" ] } ] } ], "locations": [ { "location": 31082, "action": "INSERT", "locationType": "S" } ], "customAttributes": [ { "id": "CND_EQ_RWD", "type": "CFA", "valueBoolean": false }, { "id": "PR_PRICE_RULE_PROMO", "type": "CFA", "valueBoolean": true }, { "id": "HOME_OFFICE_ASSOC_PROMO", "type": "CFA", "valueBoolean": false }, { "id": "WEBCAT_PROMO", "type": "CFA", "valueBoolean": true }, { "id": "REDEMPTION_CHANNELS", "type": "CFA", "values": [ "DTC" ] }, { "id": "WEB_DESKTOP", "type": "CFA", "valueBoolean": false }, { "id": "APPLYAUTOMATICALLY", "type": "CFA", "values": [ "AUTO" ] }, { "id": "HCO_BRAND", "type": "CFA", "valueBoolean": true }, { "id": "KIDS_BRAND", "type": "CFA", "valueBoolean": false }, { "id": "DTC_BIZ_PROMO_CAT", "type": "CFA", "values": [ "Members Save on Bottoms" ] }, { "id": "EMPLOYEESELIGIBLE", "type": "CFA", "valueBoolean": false }, { "id": "LOGGEDON", "type": "CFA", "valueBoolean": false }, { "id": "WEB_MOBILE", "type": "CFA", "valueBoolean": false }, { "id": "DTC_BIZ_PROMO_CAT", "type": "CFA", "values": [ "Member Pricing-HCO UK" ] }, { "id": "ALL_CHANNELS", "type": "CFA", "valueBoolean": true }, { "id": "APP", "type": "CFA", "valueBoolean": false }, { "id": "EXTERNALNAME", "type": "CFA", "values": [ "Members Save on Bottoms - HCO UK" ] }, { "id": "ANF_BRAND", "type": "CFA", "valueBoolean": false }, { "id": "GH_BRAND", "type": "CFA", "valueBoolean": false }, { "id": "PHYSICAL_STORE", "type": "CFA", "valueBoolean": false }, { "id": "ALLOW_UNIQUE_CD_REPLEN", "type": "CFA", "valueBoolean": false }, { "id": "EXTERNALNAME", "type": "CFA", "values": [ "Member Pricing-HCO UK" ] }, { "id": "CATEGORY", "type": "CFA", "values": [ "ECOM" ] }, { "id": "STORES_BIZ_PROMO_CAT", "type": "CFA", "values": [ "NA" ] }, { "id": "CALL_CENTER", "type": "CFA", "valueBoolean": false } ], "attributes": { "priceType": "PROMOTION", "createTimestamp": "2024-10-29T13:02:34.000-04:00", "createUserId": "REST_USER", "approvalId": "REST_USER", "lastUpdateProgramId": "PCS", "lastUpdateTimestamp": "2024-10-29T13:07:35.000-04:00", "lastUpdateUserId": "REST_USER", "emergencyInd": false, "promoTypeCode": "TP" } }

Spec

[ { "operation": "modify-overwrite-beta", "spec": { "tiers": { "*": { "conditions": { "*": { "conditionsMerch": { "*": { "webCategoryId": ["=toString", "notfound"] } } } } } } } }, { "operation": "shift", "spec": { "description": "priceRuleName", "startDateTime": "startDate", "endDateTime": "enddate", "locations": { "0": { "location": "storeLocation" } }, "customAttributes": { "*": { "id": { "PR_PRICE_RULE_PROMO": { "@(2,valueBoolean)": "priceRulePromo" }, "ANF_BRAND": { "@(2,valueBoolean)": { "true": { "#ANF": "brand" } } }, "KIDS_BRAND": { "@(2,valueBoolean)": { "true": { "#KIDS": "brand" } } }, "HCO_BRAND": { "@(2,valueBoolean)": { "true": { "#HCO": "brand" } } }, "GH_BRAND": { "@(2,valueBoolean)": { "true": { "#GH": "brand" } } } } } }, "tiers": { "*": { "priceRuleAttributes": { "*": { "id": { "PR_EMPTY_PATH_NAME": { "@(2,values[0])": "emptyPathName" }, "PR_PATH_NAME": { "@(2,values[0])": "pricePaths.paths.[&5].pathName" }, "PR_TARGET_SEGMENT": { "@(2,values)": "pricePaths.paths.[&5].customerCondition[#2].targetSegments" }, "PR_CURR_CD": { "@(2,values[0])": "pricePaths.paths.[&5].currencyCondition[#2].currency" } } } }, "conditions": { "*": { "conditionsMerch": { "*": { "merchandiseLevel": { "5": { "@(2,excludeInd)": { "0": { "@(4,keyItemNum)": "pricePaths.paths[&9].catalogCondition.products.yes.productList[]" }, "1": { "@(4,keyItemNum)": "pricePaths.paths[&9].catalogCondition.products.no.productList[]" } } }, "*": { "@(2,webCategoryId)": { "|notfound": { "@(4,excludeInd)": { "0": { "@(6,item)": "pricePaths.paths[&9].catalogCondition.products.yes.productList[]" }, "1": { "@(6,item)": "pricePaths.paths[&9].catalogCondition.products.no.productList[]" } } }, "*": { "@(4,excludeInd)": { "0": { "@(6,webCategoryId)": "pricePaths.paths[&9].catalogCondition.category.yes.categoryList[]" }, "1": { "@(6,webCategoryId)": "pricePaths.paths[&9].catalogCondition.category.no.categoryList[]" } } } } } } } } } }, "rewards": { "*": { "changeType": "pricePaths.paths.[&3].calculatePrice[&1].calculationType", "changeAmount": "pricePaths.paths.[&3].calculatePrice[&1].amount", "changePercent": "pricePaths.paths.[&3].calculatePrice[&1].percentage", "@(4,promotionId)": "pricePaths.paths.[&3].calculatePrice[&1].RPMPromoId", "@(4,offerId)": "pricePaths.paths.[&3].calculatePrice[&1].RPMCompId", "@(2,priceRuleAttributes)": { "*": { "id": { "PR_CURR_CD": { "@(2,values[0])": "pricePaths.paths.[&7].calculatePrice[&4].currency" } } } } } } } } } }, { "operation": "shift", "spec": { "*": "&", "pricePaths": { "paths": { "*": { "*": "&3.&2[&1].&", "catalogCondition": { "*": { "*": { "*": "&6.&5[&4].&3.&2[#2].&", "$": "&6.&5[&4].&3.&2[#2].include" } } } } } } } } ]

Re: JOLT Nested Array

SAMSAL — Wed, 30 Oct 2024 02:51:48 GMT

Hi @Syed0000 ,

Do you mind posting what the expected output looks like. Also please provide description of what are you trying to achieve with the transformation?

Re: JOLT Nested Array

Syed0000 — Wed, 30 Oct 2024 14:50:56 GMT

I am trying to group category & product list with yes/no condition.

if merchandiseLevel = 5 then it will consider keyItemNum and populate productList.

if merchandiseLevel != 5 then check
if webCategoryId is exists then consider webCategoryId and populate categoryList
else consider item and populate productList.

Expected output

{ "priceRuleName" : "Price Rule wIncl and Excl - HCO UK", "startDate" : "2024-10-30T00:00:00.000Z", "enddate" : "2024-10-31T23:59:00.000Z", "storeLocation" : 31082, "priceRulePromo" : true, "brand" : "HCO", "emptyPathName" : "No Everbody Pricing", "pricePaths" : { "paths" : [ { "pathName" : "Members Save on HCO AE", "customerCondition" : [ { "targetSegments" : [ "LOGIN" ] } ], "catalogCondition" : { "products" : [ { "include" : "yes", "productList" : [ "KIC_123444" ] }, { "include" : "no", "productList" : [ "KIC_1111111" ] } ], "category" : [ { "include" : "yes", "categoryList" : [ "WEBCTGY_85265"] }, { "include" : "no", "categoryList" : [ "WEBCTGY_166330"] } ] }, "calculatePrice" : [ { "RPMPromoId" : 569, "RPMCompId" : 2173, "calculationType" : 2, "amount" : 30, "percentage" : 0 } ] }, { "pathName" : "Members Save on Bottoms", "customerCondition" : [ { "targetSegments" : [ "LOGIN" ] } ], "catalogCondition" : { "products" : [ { "include" : "no", "productList" : [ "KIC_1111111" ] }, { "include" : "yes", "productList" : [ "KIC_123444" ] } ], "category" : [ { "include" : "yes", "categoryList" : [ "WEBCTGY_166330"] }, { "include" : "no", "categoryList" : [ "WEBCTGY_85265"] } ] }, "calculatePrice" : [ { "RPMPromoId" : 569, "RPMCompId" : 2173, "calculationType" : 2, "amount" : 20, "percentage" : 0 } ] } ] } }

Re: JOLT Nested Array

SAMSAL — Thu, 31 Oct 2024 23:55:40 GMT

Hi @Syed0000 ,

Sorry for the delay. You json is quite complex and very nested which make jolt very hard to write. You probably know your data very well and my recommendation before writing jolt for such Json is that you try to simplify it first by stripping un need blocks and\or flattening the structure where the transformation is not going to be on the same nested level.

I honestly tried but when the jolt got so nested it got harder to reference the upper fields and upper array indexes to maintain the same array position and then maintain the required grouping of fields, and may be that is the draw back of using jolt spec in these scenarios (without prior simplification of course :). This leads me to the second option, which I think I have mentioned to you in prior post regarding using JSLT Transformation instead.

JSLT is better option in these cases because you can traverse the structure much easier , and since you have some condition on how to set fields and values this also would be easier to achieve with some expression language like if-else .

For example if we take the transformation to create catalogCondition strucutre which seems to the most complex, here is how the jstl looks like:

let paths ={ "paths":[for(.tiers) { for(.conditions) "catalogCondition": { "category": [for(.conditionsMerch) { "include": if(.excludeInd==0) "yes" else "no", "categoryList":[.webCategoryId] } if(.merchandiseLevel!=5 and not(.keyItemNum))] } + { "products" : [for(.conditionsMerch) { "include": if(.excludeInd==0) "yes" else "no", "productList":[.keyItemNum] } if(not(.webCategoryId))] } } ] } { "pricePaths": $paths }

I know this might look a little intimidating at beginning but its not near as bad as when you try to do it with jolt.

I understand that jstl is bit of learning curve but I believe it would save you tons of time long term specially when dealing with long, complex and nested Json transformation.

I know this is not the answer you were looking for but hopefully this can help you when facing other json transformation challenges in the future.

Good luck.