Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi - Need help with JOLT-Syntax JoltTransformJSON - data manipulation possible?

NiFi - Need help with JOLT-Syntax JoltTransformJSON - data manipulation possible?

Rising Star

Hi, I need some help with JOLT-syntax - once again.


So far I processed only individual records in json-format.

If there was a timestamp I putted it into an attribute and managed with expression language to get the right format and handle empty timestamps with set to 'null'.

Just like that I were able to work with escapeJson() to handle control-chars. These attributes I was able to use in my JOLT.


Now I have to deal with nested structures. Working with attributes like above isn't possible anymore.


JSON-Input

{
  "nummer": "319",
  "name": "Dieselverbrauch Test IJ - NiFi TEST\n2019-03-27",
  "stand": "20190604094517",
  "table": [
    {
      "zn": 1,
      "dieselart": "977",
      "einheit": "178",
      "verbrauch": 12.33,
      "gueltigab": "20190601",
      "staffelbis": 10.5
    },
    {
      "zn": 2,
      "dieselart": "977",
      "einheit": "178",
      "verbrauch": 10.44,
      "gueltigab": "20190601",
      "staffelbis": 99.752
    }
  ]
}


JOLT-Specification - thanks to Matt Burgess :-)

[{
    "operation": "shift",
    "spec": {
      "nummer": "Nummer",
      "name": "Bezeichnung",
      "stand": "Stand",
      "table": {
        "*": {
          "zn": "Positionen.[#2].ZeileNr",
          "dieselart": "Positionen.[#2].ID_DieselArt",
          "einheit": "Positionen.[#2].ID_Einheit",
          "verbrauch": "Positionen.[#2].VerbrauchLiter",
          "gueltigab": "Positionen.[#2].GueltigAb",
          "staffelbis": "Positionen.[#2].StaffelBis"
        }
      }
    }
}
]


JSON-Output

{
  "Nummer" : "319",
  "Bezeichnung" : "Dieselverbrauch Test IJ - NiFi TEST\n2019-03-27",
  "Stand" : "20190604094517",
  "Positionen" : [ {
    "ZeileNr" : 1,
    "ID_DieselArt" : "977",
    "ID_Einheit" : "178",
    "VerbrauchLiter" : 12.33,
    "GueltigAb" : "20190601",
    "StaffelBis" : 10.5
  }, {
    "ZeileNr" : 2,
    "ID_DieselArt" : "977",
    "ID_Einheit" : "178",
    "VerbrauchLiter" : 10.44,
    "GueltigAb" : "20190601",
    "StaffelBis" : 99.752
  } ]
}


But the JSON-Output what I need ist:

{
  "Nummer" : "319",
  "Bezeichnung" : "Dieselverbrauch Test IJ - NiFi TEST\n2019-03-27",
  "Stand" : "2019-06-04 09:45:17.000",
  "Positionen" : [ {
    "ZeileNr" : 1,
    "ID_DieselArt" : "977",
    "ID_Einheit" : "178",
    "VerbrauchLiter" : 12.33,
    "GueltigAb" : "2019-06-01",
    "StaffelBis" : 10.5
  }, {
    "ZeileNr" : 2,
    "ID_DieselArt" : "977",
    "ID_Einheit" : "178",
    "VerbrauchLiter" : 10.44,
    "GueltigAb" : "2019-06-01",
    "StaffelBis" : 99.752
  } ]
}


So how can I do such manipulations now? Must I change the json content before JOLT (and if so - how?) or is it possible to do this within JOLT?


Found this: https://community.hortonworks.com/questions/207986/nifi-joltjsontransform-spec-help-needed.html

Because I don't know whether the order of the Input is arbitrary is using a script really the only solution?

Any help is very appriciated!