Reply
New Contributor
Posts: 2
Registered: ‎10-26-2017

schema incompatible issue in impala

We are facing incompatible schema issue when there is change in file schema (when new fields being added to file) while querying on Impala but it works fine on Beeline. Please help us in resolving this issue.

Cloudera Employee
Posts: 443
Registered: ‎03-23-2015

Re: schema incompatible issue in impala

What's the issue in impala? Impala don't see the new columns? Did you update table to include the new columns through Beeline? Have you validated metadata in Impala?
New Contributor
Posts: 2
Registered: ‎10-26-2017

Re: schema incompatible issue in impala

we altered the table from beeline.we see the new column in the table structure but we arent able to query the hive table through impala but the same query works from beeline. yes. we have invalidated the metadata in impala .

Cloudera Employee
Posts: 443
Registered: ‎03-23-2015

Re: schema incompatible issue in impala

Hi Ullas,

Can you please share the DDL of the table from both Impala and Hive and also the ALTER command you used to update schema?

Thanks
Highlighted
New Contributor
Posts: 2
Registered: ‎10-05-2017

Re: schema incompatible issue in impala

The solution to your issue likely depends on what type of files back your table, but if you are using parquet, this option is probably what you are looking for:

 

 

 

set PARQUET_FALLBACK_SCHEMA_RESOLUTION=name;

 

https://www.cloudera.com/documentation/enterprise/5-8-x/topics/impala_parquet.html#parquet_schema_ev...

 

 

The issue is that by default, impala expects every entry in the parquet schema to be at the same ordinal position. If you add to your schema anywhere but the end impala starts throwing errors. The option above makes impala flexible about the ordinal positions within the parquet files.

 

Announcements