question Re: getting null from JoltTransformJson in nifi while converting a plain json to nested json in Support Questions

getting null from JoltTransformJson in nifi while converting a plain json to nested json

RanjitMohapatra — Tue, 22 Oct 2024 06:49:47 GMT

Hi,

I am using JoltTransformJson to convert a plain json to nested json in Nifi and getting null in the output. The same i/p and spec is working in Jolt demo. Below are the input and the spec
Input:

{
"ts" : "2024-09-22T12:03:13.000Z",
"sourceDeviceId" : 4321,
"sourceName" : "light1",
"sourceType" : "light",
"locationLat" : 201.34,
"locationLon" : 101.5,
"locationId" : 1,
"recordDate" : "2024-09-22T12:03:13.000Z",
"_class" : "com.prutech.smartcity.entity.LightEntity",
"_id" : "66f27c487bd8652abecdb9c7",
"status" : "INACTIVE",
"power" : 0,
"current" : 0,
"intensity" : 100,
"voltage" : 0
}

Jolt Spec:

{

"operation": "shift",

"spec":{

"ts":"ts",

"sourceDeviceId":"source.deviceId",

"sourceName":"source.name",

"sourceType":"source.type",

"locationLat":"location.lat",

"locationLon":"location.lon",

"locationId":"location.id",

"recordDate":"recordDate",

"_class":"_class",

"status":"status",

"power":"power",

"current":"current",

"intensity":"intensity",

"voltage":"voltage"

}

Please help.

Re: getting null from JoltTransformJson in nifi while converting a plain json to nested json

drewski7 — Tue, 22 Oct 2024 14:59:47 GMT

@RanjitMohapatra

NiFi expects the Jolt specification to be provided as an array of objects if it's set chain. This is typical when using JoltTransformJSON because it supports multiple transformations chained together.

Also, if you click on your JoltTransformJSON and click on the advanced option -

It will bring up a nice Jolt Spec tester for you that is consistent with how NiFi will handle the transformation -

Solution is just wrapping it in an array.

[
    {
          "operation": "shift",
          "spec": {
               "ts": "ts",
               "sourceDeviceId": "source.deviceId",
               "sourceName": "source.name",
               "sourceType": "source.type",
               "locationLat": "location.lat",
               "locationLon": "location.lon",
               "locationId": "location.id",
               "recordDate": "recordDate",
               "_class": "_class",
               "status": "status",
               "power": "power",
               "current": "current",
               "intensity": "intensity",
               "voltage": "voltage"
           }
      }
]

Please accept the solution!

Re: getting null from JoltTransformJson in nifi while converting a plain json to nested json

DianaTorres — Fri, 25 Oct 2024 17:01:49 GMT

@RanjitMohapatra

Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.

Re: getting null from JoltTransformJson in nifi while converting a plain json to nested json

kavyashree — Wed, 12 Mar 2025 00:29:53 GMT

Hi,

I have a similar issue. When I use "Advanced" to check what output nifi would provide, I get the output there, but when I check the queue after the JoltTransformJSON has processed the output, it says null. I changed "Jolt Transform DSL" from "chain" to "default", as it eliminated the space that was there between the square brackets and the array elements in an array. This is my Jolt specification -

[
{
"operation": "shift",
"spec": {
"uid": "uid",
"location_name": "name",
"location_city": "city",
"age_min": "age_min",
"age_max": "age_max",
"firstdate_begin": "begin",
"lastdate_end": "end",
"location_coordinates": {
"lon": "location[0]",
"lat": "location[1]"
}
}
}
]

and this is the expected output -

{
"uid": "54570223",
"name": "Théâtre Plaza",
"city": "Montréal",
"age_min": null,
"age_max": null,
"begin": "2024-03-23T23:30:00+00:00",
"end": "2024-03-24T01:00:00+00:00",
"location": [ -73.603196, 45.536315 ]
}

EDIT -

I edited my jolt specification, and it looks like this now-

[
{
"operation": "shift",
"spec": {
"*": {
"uid": "uid",
"location_name": "location_name",
"location_city": "location_city",
"age_min": "age_min",
"age_max": "age_max",
"firstdate_begin": "firstdate_begin",
"lastdate_end": "lastdate_end",
"location_coordinates": {
"lon": "location[0]",
"lat": "location[1]"
}
}
}
}
]

I am getting the desired output -
{
"uid": "70029814",
"location_name": "Sanctuaire du Saint-Sacrement",
"location_city": "Montreal",
"age_min": null,
"age_max": null,
"firstdate_begin": "2019-04-13T18:00:00+00:00",
"lastdate_end": "2019-04-14T10:00:00+00:00",
"location": [-73.581721, 45.525243]
}

But the issue now is that after processing, the output looks something like this -

{
"uid" : [ "54570223" ],
"location_name" : [ "Théâtre Plaza" ],
"location_city" : [ "Montréal" ],
"age_min" : [ 16 ],
"age_max" : [ 99 ],
"firstdate_begin" : [ "2024-03-23T23:30:00+00:00" ],
"lastdate_end" : [ "2024-03-24T01:00:00+00:00" ],
"location" : [ [ -73.603196 ], [ 45.536315 ] ]
}

Working on keeping only the location as arrays. 😅😅😅