Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

create SQL table from AVRO Schema

Contributor

Hello,

Using InferAvroSchema I got an avro Schema of my file. I want to create at able in PostregSql using this avroSchema. Which processor I have to use and connect to InferAvroSchema to create my table.

Thank you!

5 REPLIES 5

How big is your avro Schema? There is always the workaround of hard-coding the column names to the SQL query.

Contributor

for my test file I have 10 columns but I want the work to be done independently of this file. For example, I want to use getfile-> infeAvroSchema-> Create the table in Postgresl and then use PutdatabaseRecod. Once I have my schema generated I do not know what exactly to use to create the table.

Contributor

Sorry, can't really help you with that. But there is a processor for executing SQL statements. Maybe that might help. @Amira khalifa

@Amira khalifa

There is no standard processor capable to address your requirement. You would have to build a custom processor to generate the DDL or use ExecuteSQL or ExecuteScript processor. Anyhow, keep in mind that avro data types are not an exact match with Postgresql data types to not mention that your avro may be hierarchical. For that situation I suggest you to follow the steps: ConvertAvroToJson -> FlattenJson -> ConvertJsonToSQL -> ExecuteSQL. This is just one approach, you can do it in several ways including using Record processors.



Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.