Removing Null Values using jolt in a nested JSON response

AmiJhones — Wed, 03 Aug 2022 16:07:08 GMT

Hi, I'm new to jolt and am trying to remove null values from a nested json using jolt.

Example JSON:

{
"field1": {
"subfield1": null
},
"field2": {
"subfield21": "528010000072312",
"subfield22": null
},
"field3": {
"field31": null,
"field32": null,
"field33": {
"field331": null,
"field332": null
}
},
"field4": {
"field41": null,
"field42": "UNKNOWN",
"field43": "NONE",
"field44": {
"field441": {
"field4411": null,
"field4412": null
}
}
},
"field5": {
"field51": null,
"field52": {
"field521": null,
"field522": null
},
"field53": {
"field531": null,
"field532": {
"field5321": null
},
"field533": {
"field5331": "0"
}
}
}

JOLT SPEC:

[
{
"operation": "shift",
"spec": {
"field1": {
"*": {
"*": {
"@1": "&3.&2"
}
}
},
"field2": {
"*": {
"*": {
"@1": "&3.&2"
}
}
},
"field3": {
"*": {
"*": {
"@1": "&3.&2"
},
"field33": {
"*": {
"@1": "&3.&2"
}
}
}
},
"field4": {
"*": {
"*": {
"@1": "&3.&2"
},
"field44": {
"field441": {
"*": {
"@1": "&3.&2"
}
}
}
}
},
"field5": {
"*": {
"*": {
"@1": "&3.&2"
},
"field53": {
"*": {
"*": {
"@1": "&3.&2"
}
}
}
}
},
"*": {
"*": {
"@1": "&2"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
}
]

OUTPUT:

{
"field2" : {
"subfield21" : "528010000072312"
},
"field3" : {
"field33" : [ { }, { } ]
},
"field4" : {
"field42" : "UNKNOWN",
"field43" : "NONE",
"field44" : {
"field441" : { }
}
},
"field5" : {
"field52" : [ { }, { } ],
"field53" : [ {
"field532" : { },
"field533" : {
"field5331" : "0"
}
}, {
"field532" : { },
"field533" : {
"field5331" : "0"
}
}, {
"field532" : { },
"field533" : {
"field5331" : "0"
}
} ]
}
}

EXPECTED OUTPUT:

{
"field2": {
"subfield21": "528010000072312"
},
"field4": {
"field42": "UNKNOWN",
"field43": "NONE"
},
"field5": {
"field53": {
"field533": {
"field5331": "0"
}
}
}

QUESTIONS:

1. How can I completely remove an empty node in json using jolt?

2. Why is the values for "field5" being repeated?

Would very much appreciate help with Jolt 🙂

Re: Removing Null Values using jolt in a nested JSON response

araujo — Wed, 03 Aug 2022 23:24:18 GMT

@AmiJhones ,

I've answer a similar question before: https://community.cloudera.com/t5/Support-Questions/Json-Jolt-to-Remove-All-Nulls-from-Json/td-p/336008

Cheers,

André

question Re: Removing Null Values using jolt in a nested JSON response in Support Questions

Removing Null Values using jolt in a nested JSON response

Re: Removing Null Values using jolt in a nested JSON response