Problem with JoltTransformation in NIFI

Hi @MWM ,

There are couple of issues I noticed regarding the input json and the spec:

1- Regarding the "paycheck" value "3456,98" , I noticed there is coma (,) is that correct? if that is the case , Im not sure the conversion will work in this case.  You have to probably replace the coma with dot (.) and then convert toDouble. It takes few steps to do this in jolt since there is no string replace function using split, join as follows:

,
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      ....
      "paycheckArray": "=split('[,]',@(1,paycheck))",
      "paycheckJoin": "=join('.',@(1,paycheckArray))",
      "paycheck": "=toDouble(@(1,paycheckJoin))",
      ....
    }
}
,
  {
    "operation": "remove",
    "spec": {
      "paycheckArray": "",
      "paycheckJoin": ""
    }
  }

2- Regarding the boolean value "isOfAge" , since you have moved it under person object, the spec would as follows:

{
    "operation": "modify-overwrite-beta",
    "spec": {
      ...
      "person": {
        "isOfAge": "=toBoolean(@(1,isOfAge))"
      }
     ....
    }
}

3- Regarding converting the date value "joiningDate" to number , this is tricky and I dont think you can do it via Jolt, you either have to extract it using EvaluateJsonPath as an attribute then do update attribute using expression language to do the proper formatting and conversion. Since JoltTransformation spec allows expression language , you can reference this attribute in the modify spec. here is an example:

https://stackoverflow.com/questions/76678896/jolt-how-to-convert-datetimestamp-to-epoch-on-dynamic-json-keys

Another option is to use UpdateRecord to do the proper formatting, conversion for all attributes that require such thing and then just use basic shift spec to re arrange values accordingly. With UpdateRecord you need to use Avro Schema to define the proper types in the JsonRecordSetWriter.

UpdateRecord:

JsonRecordSetWriter:

AvroSchema in the Schema Text Property:

{
  "type": "record",
  "name": "nifi-record",
  "fields":
  [
    { "name": "id", "type": "string"},
    { "name": "system", "type": "string"},
    { "name": "sourceSystem", "type": "string"},
    { "name": "name", "type": "string"},
	{ "name": "lastName", "type": "string"},
	{ "name": "mail", "type": "string"},
	{ "name": "country", "type": "string"},
	{ "name": "city", "type": "string"},
	{ "name": "street", "type": "string"},
	{ "name": "building", "type": "string"},
	{ "name": "local", "type": "string"},
	{ "name": "paycheck", "type": "double"},
	{ "name": "joiningDate", "type": { "type":"int", "logicalType":"date" }},
    { "name": "isOfAge", "type": ["boolean","null"]}
  ]
}

If you find this helpful please accept solution.

Thanks