We are facing incompatible schema issue when there is change in file schema (when new fields being added to file) while querying on Impala but it works fine on Beeline. Please help us in resolving this issue.
we altered the table from beeline.we see the new column in the table structure but we arent able to query the hive table through impala but the same query works from beeline. yes. we have invalidated the metadata in impala .
The solution to your issue likely depends on what type of files back your table, but if you are using parquet, this option is probably what you are looking for:
The issue is that by default, impala expects every entry in the parquet schema to be at the same ordinal position. If you add to your schema anywhere but the end impala starts throwing errors. The option above makes impala flexible about the ordinal positions within the parquet files.