Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

From JSON to get Avro Schema

avatar
Explorer

Hi guys! I have a question about Avro schema.

My input flowfile are in json format and I want to convert it into xml. The method I use is using ConvertRecord Processor. In terms of ConvertRecord Processor, I use JsonTreeReader as record reader and XMLRecordSetWriter as record writer. Following picture is the properties of JsonTreeReader I set it up. 

Here is the json I want to convert:

 

{
 "article":{
  "article-meta": {
   "art_id": "me08001",
   "title" : [ {
    "lang" : "en",
    "title" : "A"
    }, {
    "lang" : "hr",
    "title" : "B"
   } ]
  },
  "journal-meta": {
   "journal_id": "me9",
   "type" : "journal"
  }
}

 

If I have a lot of flowfile that contain this kind of format Json file, what Avro schema should I put inside AvroSchemaRegistry. I try a lot of online converter and it doesn't satisfy AvroSchemaRegistry when I do validation. I need the Avro schema so that the ConvertRecord can read my JSON flowfile.

Thank you in advance

 

 

 

Screenshot 2024-01-10 at 1.24.34 PM.png

Screenshot 2024-01-10 at 1.24.00 PM.png

1 ACCEPTED SOLUTION

avatar
Master Mentor

@FrankHaha 

Have you tried using the "Infer Schema" Schema Access Strategy in the JsonTreeReader 1.24.0 controller services instead of fetching schema from AvroSchemaRegistry?   

Another option would be to use the ExtractRecordSchema 1.24.0 processor along with JsonTreeReader 1.24.0 controller services configured with "Infer Schema" Schema Access Strategy" to output the schema into a FlowFile Attribute "avro.schema".  You can then get the produced schema from that FlowFile Attribute to add to your AvroSchemaRegsitry for future use.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

View solution in original post

1 REPLY 1

avatar
Master Mentor

@FrankHaha 

Have you tried using the "Infer Schema" Schema Access Strategy in the JsonTreeReader 1.24.0 controller services instead of fetching schema from AvroSchemaRegistry?   

Another option would be to use the ExtractRecordSchema 1.24.0 processor along with JsonTreeReader 1.24.0 controller services configured with "Infer Schema" Schema Access Strategy" to output the schema into a FlowFile Attribute "avro.schema".  You can then get the produced schema from that FlowFile Attribute to add to your AvroSchemaRegsitry for future use.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt