Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

create SQL table from AVRO Schema

Highlighted

create SQL table from AVRO Schema

Contributor

Hello,

Using InferAvroSchema I got an avro Schema of my file. I want to create at able in PostregSql using this avroSchema. Which processor I have to use and connect to InferAvroSchema to create my table.

Thank you!

5 REPLIES 5

Re: create SQL table from AVRO Schema

How big is your avro Schema? There is always the workaround of hard-coding the column names to the SQL query.

Highlighted

Re: create SQL table from AVRO Schema

Contributor

for my test file I have 10 columns but I want the work to be done independently of this file. For example, I want to use getfile-> infeAvroSchema-> Create the table in Postgresl and then use PutdatabaseRecod. Once I have my schema generated I do not know what exactly to use to create the table.

Highlighted

Re: create SQL table from AVRO Schema

Contributor
Highlighted

Re: create SQL table from AVRO Schema

Sorry, can't really help you with that. But there is a processor for executing SQL statements. Maybe that might help. @Amira khalifa

Highlighted

Re: create SQL table from AVRO Schema

@Amira khalifa

There is no standard processor capable to address your requirement. You would have to build a custom processor to generate the DDL or use ExecuteSQL or ExecuteScript processor. Anyhow, keep in mind that avro data types are not an exact match with Postgresql data types to not mention that your avro may be hierarchical. For that situation I suggest you to follow the steps: ConvertAvroToJson -> FlattenJson -> ConvertJsonToSQL -> ExecuteSQL. This is just one approach, you can do it in several ways including using Record processors.



Don't have an account?
Coming from Hortonworks? Activate your account here