getting null from JoltTransformJson in nifi while converting a plain json to nested json

[#1]

Hi,

I am using JoltTransformJson to convert a plain json to nested json in Nifi and getting null in the output. The same i/p and spec is working in Jolt demo. Below are the input and the spec 
Input:

{
"ts" : "2024-09-22T12:03:13.000Z",
"sourceDeviceId" : 4321,
"sourceName" : "light1",
"sourceType" : "light",
"locationLat" : 201.34,
"locationLon" : 101.5,
"locationId" : 1,
"recordDate" : "2024-09-22T12:03:13.000Z",
"_class" : "com.prutech.smartcity.entity.LightEntity",
"_id" : "66f27c487bd8652abecdb9c7",
"status" : "INACTIVE",
"power" : 0,
"current" : 0,
"intensity" : 100,
"voltage" : 0
}

Jolt Spec:

{

"operation": "shift",

"spec":{

"ts":"ts",

"sourceDeviceId":"source.deviceId",

"sourceName":"source.name",

"sourceType":"source.type",

"locationLat":"location.lat",

"locationLon":"location.lon",

"locationId":"location.id",

         "recordDate":"recordDate",

"_class":"_class",

"status":"status",

"power":"power",

"current":"current",

"intensity":"intensity",

"voltage":"voltage"

}

}

Please help.

[#2 · Accepted solution]

@RanjitMohapatra 

NiFi expects the Jolt specification to be provided as an array of objects if it's set chain. This is typical when using JoltTransformJSON because it supports multiple transformations chained together. 

Also, if you click on your JoltTransformJSON and click on the advanced option - 

It will bring up a nice Jolt Spec tester for you that is consistent with how NiFi will handle the transformation -

Solution is just wrapping it in an array.

[
    {
          "operation": "shift",
          "spec": {
               "ts": "ts",
               "sourceDeviceId": "source.deviceId",
               "sourceName": "source.name",
               "sourceType": "source.type",
               "locationLat": "location.lat",
               "locationLon": "location.lon",
               "locationId": "location.id",
               "recordDate": "recordDate",
               "_class": "_class",
               "status": "status",
               "power": "power",
               "current": "current",
               "intensity": "intensity",
               "voltage": "voltage"
           }
      }
]

Please accept the solution! 

[#3]

@RanjitMohapatra 

Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  Thanks.

[#4]

Hi,

I have a similar issue. When I use "Advanced" to check what output nifi would provide, I get the output there, but when I check the queue after the JoltTransformJSON has processed the output, it says null. I changed "Jolt Transform DSL" from "chain" to "default", as it eliminated the space that was there between the square brackets and the array elements in an array. This is my Jolt specification -

[
{
"operation": "shift",
"spec": {
"uid": "uid",
"location_name": "name",
"location_city": "city",
"age_min": "age_min",
"age_max": "age_max",
"firstdate_begin": "begin",
"lastdate_end": "end",
"location_coordinates": {
"lon": "location[0]",
"lat": "location[1]"
}
}
}
]

and this is the expected output -

{
"uid": "54570223",
"name": "Théâtre Plaza",
"city": "Montréal",
"age_min": null,
"age_max": null,
"begin": "2024-03-23T23:30:00+00:00",
"end": "2024-03-24T01:00:00+00:00",
"location": [ -73.603196, 45.536315 ]
}

EDIT -

I edited my jolt specification, and it looks like this now-

[
{
"operation": "shift",
"spec": {
"*": {
"uid": "uid",
"location_name": "location_name",
"location_city": "location_city",
"age_min": "age_min",
"age_max": "age_max",
"firstdate_begin": "firstdate_begin",
"lastdate_end": "lastdate_end",
"location_coordinates": {
"lon": "location[0]",
"lat": "location[1]"
}
}
}
}
]

I am getting the desired output -
{
"uid": "70029814",
"location_name": "Sanctuaire du Saint-Sacrement",
"location_city": "Montreal",
"age_min": null,
"age_max": null,
"firstdate_begin": "2019-04-13T18:00:00+00:00",
"lastdate_end": "2019-04-14T10:00:00+00:00",
"location": [-73.581721, 45.525243]
}

But the issue now is that after processing, the output looks something like this -

{
"uid" : [ "54570223" ],
"location_name" : [ "Théâtre Plaza" ],
"location_city" : [ "Montréal" ],
"age_min" : [ 16 ],
"age_max" : [ 99 ],
"firstdate_begin" : [ "2024-03-23T23:30:00+00:00" ],
"lastdate_end" : [ "2024-03-24T01:00:00+00:00" ],
"location" : [ [ -73.603196 ], [ 45.536315 ] ]
}

Working on keeping only the location as arrays. 😅😅😅