question NiFi - converting JSON array into Avro Map in Support Questions

NiFi - converting JSON array into Avro Map

migu — Wed, 08 Jun 2022 14:51:41 GMT

Hello,

On our use case, we're receiving data in JSON format and it must be converted into Avro using a schema that has X and Y includes mandatory fields and everything else (Z and W) should go into a Custom map.

Here is our Avro schema:

"fields": [
{"name": "X", "type": "long"},
{"name": "Y", "type": "string"},
{"name": "Custom", "type": { "type": "map", "values": "string" } }
]

And an example of data coming in:

{

"X":123,

"Y":"ABC",

"Z":"zzz",

"W":"www"

}

We have implemented a JoltTransform to do the needed transformation to get Z and W into a Custom array.

Now the issue is on doing the conversion from JSON into Avro since we get an error stating it cannot convert an java.lang.Object into a Map.

How can we properly process this JSON and put that array into a Map so it can be properly stored using that Avro schema?

Thank you.

Best regards,

Miguel

Re: NiFi - converting JSON array into Avro Map

araujo — Thu, 09 Jun 2022 03:08:02 GMT

@migu ,

What's your NiFi version and what's your JOLT transformation?

This should work, provided that your JOLTed JSON is correct.

Please see the attached flow, which works fine for me.

Cheers,

André

Re: NiFi - converting JSON array into Avro Map

DianaTorres — Mon, 13 Jun 2022 15:52:57 GMT

@migu Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. If you are still experiencing the issue, can you provide the information @araujo has requested? Thanks