Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

When creating a Hive table I am encountering SemanticException [Error 10043] erro

Highlighted

When creating a Hive table I am encountering SemanticException [Error 10043] erro

New Contributor

I am using Hive, CDH-5.4.7-1.cdh5.4.7.p0.3, to create a Parquet format table using literal Avro schema as follows


hive> create external table if not exists datamodel.bcdfsegway_Test1
> STORED AS PARQUET
> LOCATION 'hdfs://nameservice1/project/segway/'
> TBLPROPERTIES ('avro.schema.literal'='{
> "type" : "record",
> "name" : "Test1",
> "namespace" : "com.company.avro.generated",
> "doc" : "some comments.",
> "fields" : [ {
> "name" : "executionInfoDetails",
> "type" : [ "null", "string" ]
> }, {
> "name" : "batchTimestamp",
> "type" : [ "null", "int" ]
> }, {
> "name" : "recordTimestamp",
> "type" : [ "null", "int" ]
> }]
> }')
> ;
FAILED: SemanticException [Error 10043]: Either list of columns or a custom serializer should be specified
hive>

 

Is this the correct syntax for my purpose?

 

thanks,

Bruce 

1 REPLY 1

Re: When creating a Hive table I am encountering SemanticException [Error 10043] erro

Explorer

list of columns are required for parquet file format. One can't use AVRO schema for defining columns of a parquet table. You can try below.

 

1) specify list of columns of your external table.

2) create an avro table with your AVRO schema and then use CTAS for your Parquet table creation.

 

Cloudera Team,

 

Please guide in creating a custom serializer or using any available serializer to create a Parquet table based on avro schema.

OR

If there is any other best possible way.

 

Best regards,

Don't have an account?
Coming from Hortonworks? Activate your account here