- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
I want to Convert JsonSchema to AvroSchema & HiveDDL
- Labels:
-
Apache Hive
Created on ‎05-28-2018 08:39 AM - edited ‎09-16-2022 06:16 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I want to convert JSON schema to Avro schema and hive DDL and then insert it into the table.
The schemas I have are a multi-referenced chained schemas. I wrote a processor which validates JSON agaisnt JSON schema but Im unable to find any library for the above-mentioned tasks.
Two questions.
Is converting Json schema to Avro is a good idea? because currently I've to write avro schema by hand. i want to automate it.
second. If yes, then how to do it or how should I proceed?
Any help would be appreciated.
Thanks
Created ‎05-28-2018 05:10 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Have a look at this if you need to by pass Avro for Hive DDL:
https://github.com/tspannhw/nifi-convertjsontoddl-processor
If you need to convert JSON to ORC (for Hive) Avro will be required. You will need to write/manage Avro schemas (recommended will be to use Schema Registry for that). Alternatively you can use Infer Avro schema to detect incoming schema from JSON but it may not be 100% correct all the time.
