Support Questions

Find answers, ask questions, and share your expertise

NiFi - converting JSON array into Avro Map

avatar
Explorer

Hello,

 

On our use case, we're receiving data in JSON format and it must be converted into Avro using a schema that has X and Y includes mandatory fields and everything else (Z and W) should go into a Custom map.

 

Here is our Avro schema:

"fields": [
{"name": "X", "type": "long"},
{"name": "Y", "type": "string"},
{"name": "Custom", "type": { "type": "map", "values": "string" } }
]

 

And an example of data coming in:

{

"X":123,

"Y":"ABC",

"Z":"zzz",

"W":"www"

}

 

We have implemented a JoltTransform to do the needed transformation to get Z and W into a Custom array.

Now the issue is on doing the conversion from JSON into Avro since we get an error stating it cannot convert an java.lang.Object into a Map.

 

How can we properly process this JSON and put that array into a Map so it can be properly stored using that Avro schema?

 

Thank you.

Best regards,

Miguel

1 ACCEPTED SOLUTION

avatar
Super Guru

@migu ,

 

What's your NiFi version and what's your JOLT transformation?

This should work, provided that your JOLTed JSON is correct.

 

Please see the attached flow, which works fine for me.

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

View solution in original post

2 REPLIES 2

avatar
Super Guru

@migu ,

 

What's your NiFi version and what's your JOLT transformation?

This should work, provided that your JOLTed JSON is correct.

 

Please see the attached flow, which works fine for me.

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Community Manager

@migu Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. If you are still experiencing the issue, can you provide the information @araujo has requested? Thanks


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: