Skip to main content
RanjitMohapatra
RanjitMohapatra
Bronze ⭐️⭐️⭐️
October 22, 2024
Solved

getting null from JoltTransformJson in nifi while converting a plain json to nested json

Hi,

I am using JoltTransformJson to convert a plain json to nested json in Nifi and getting null in the output. The same i/p and spec is working in Jolt demo. Below are the input and the spec 
Input:

{
"ts" : "2024-09-22T12:03:13.000Z",
"sourceDeviceId" : 4321,
"sourceName" : "light1",
"sourceType" : "light",
"locationLat" : 201.34,
"locationLon" : 101.5,
"locationId" : 1,
"recordDate" : "2024-09-22T12:03:13.000Z",
"_class" : "com.prutech.smartcity.entity.LightEntity",
"_id" : "66f27c487bd8652abecdb9c7",
"status" : "INACTIVE",
"power" : 0,
"current" : 0,
"intensity" : 100,
"voltage" : 0
}

Jolt Spec:

{
"operation": "shift",
"spec":{
"ts":"ts",
"sourceDeviceId":"source.deviceId",
"sourceName":"source.name",
"sourceType":"source.type",
"locationLat":"location.lat",
"locationLon":"location.lon",
"locationId":"location.id",
         "recordDate":"recordDate",
"_class":"_class",
"status":"status",
"power":"power",
"current":"current",
"intensity":"intensity",
"voltage":"voltage"
}
}

Please help.

drewski7
drewski7Answer
Gold ⭐️⭐️
October 22, 2024

@RanjitMohapatra 

NiFi expects the Jolt specification to be provided as an array of objects if it's set chain. This is typical when using JoltTransformJSON because it supports multiple transformations chained together. 

Also, if you click on your JoltTransformJSON and click on the advanced option - 

drewski7_1-1729609032503.png


It will bring up a nice Jolt Spec tester for you that is consistent with how NiFi will handle the transformation -

drewski7_0-1729608992995.png

Solution is just wrapping it in an array.

[
{
"operation": "shift",
"spec": {
"ts": "ts",
"sourceDeviceId": "source.deviceId",
"sourceName": "source.name",
"sourceType": "source.type",
"locationLat": "location.lat",
"locationLon": "location.lon",
"locationId": "location.id",
"recordDate": "recordDate",
"_class": "_class",
"status": "status",
"power": "power",
"current": "current",
"intensity": "intensity",
"voltage": "voltage"
}
}
]

Please accept the solution! 

View in thread

3 replies

drewski7
drewski7Answer
Gold ⭐️⭐️
October 22, 2024

@RanjitMohapatra 

NiFi expects the Jolt specification to be provided as an array of objects if it's set chain. This is typical when using JoltTransformJSON because it supports multiple transformations chained together. 

Also, if you click on your JoltTransformJSON and click on the advanced option - 

drewski7_1-1729609032503.png


It will bring up a nice Jolt Spec tester for you that is consistent with how NiFi will handle the transformation -

drewski7_0-1729608992995.png

Solution is just wrapping it in an array.

[
{
"operation": "shift",
"spec": {
"ts": "ts",
"sourceDeviceId": "source.deviceId",
"sourceName": "source.name",
"sourceType": "source.type",
"locationLat": "location.lat",
"locationLon": "location.lon",
"locationId": "location.id",
"recordDate": "recordDate",
"_class": "_class",
"status": "status",
"power": "power",
"current": "current",
"intensity": "intensity",
"voltage": "voltage"
}
}
]

Please accept the solution! 

Translate
    DianaTorres
    Community Manager
    DianaTorres
    Community Manager
    October 25, 2024

    @RanjitMohapatra 

    Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  Thanks.

     
    Translate
    Regards, Diana Torres, Senior Community Moderator
      kavyashree
      kavyashree
      Silver ⭐️
      March 11, 2025

      Hi,

      I have a similar issue. When I use "Advanced" to check what output nifi would provide, I get the output there, but when I check the queue after the JoltTransformJSON has processed the output, it says null. I changed "Jolt Transform DSL" from "chain" to "default", as it eliminated the space that was there between the square brackets and the array elements in an array. This is my Jolt specification -

      [
      {
      "operation": "shift",
      "spec": {
      "uid": "uid",
      "location_name": "name",
      "location_city": "city",
      "age_min": "age_min",
      "age_max": "age_max",
      "firstdate_begin": "begin",
      "lastdate_end": "end",
      "location_coordinates": {
      "lon": "location[0]",
      "lat": "location[1]"
      }
      }
      }
      ]

      and this is the expected output -

      {
      "uid": "54570223",
      "name": "Théâtre Plaza",
      "city": "Montréal",
      "age_min": null,
      "age_max": null,
      "begin": "2024-03-23T23:30:00+00:00",
      "end": "2024-03-24T01:00:00+00:00",
      "location": [ -73.603196, 45.536315 ]
      }

      EDIT -

      I edited my jolt specification, and it looks like this now-

      [
      {
      "operation": "shift",
      "spec": {
      "*": {
      "uid": "uid",
      "location_name": "location_name",
      "location_city": "location_city",
      "age_min": "age_min",
      "age_max": "age_max",
      "firstdate_begin": "firstdate_begin",
      "lastdate_end": "lastdate_end",
      "location_coordinates": {
      "lon": "location[0]",
      "lat": "location[1]"
      }
      }
      }
      }
      ]

      I am getting the desired output -
      {
      "uid": "70029814",
      "location_name": "Sanctuaire du Saint-Sacrement",
      "location_city": "Montreal",
      "age_min": null,
      "age_max": null,
      "firstdate_begin": "2019-04-13T18:00:00+00:00",
      "lastdate_end": "2019-04-14T10:00:00+00:00",
      "location": [-73.581721, 45.525243]
      }

      But the issue now is that after processing, the output looks something like this -

      {
      "uid" : [ "54570223" ],
      "location_name" : [ "Théâtre Plaza" ],
      "location_city" : [ "Montréal" ],
      "age_min" : [ 16 ],
      "age_max" : [ 99 ],
      "firstdate_begin" : [ "2024-03-23T23:30:00+00:00" ],
      "lastdate_end" : [ "2024-03-24T01:00:00+00:00" ],
      "location" : [ [ -73.603196 ], [ 45.536315 ] ]
      }

      Working on keeping only the location as arrays. 😅😅😅

      Translate
      Terms & ConditionsAccessibility statement