Hive select when column name different in parquet and impala

sunilosunil — Fri, 16 Sep 2022 12:26:42 GMT

# Create parquet Impala table temp with a column a

# write parquet file using streaming applicaiton/ map reduce job call parquet schema for that

#Impala

select a from default.temp

works and returns data

#hive

select a from default.temp

returns null because it tries to reference column name from parquet schema I think and it doesn't match.

Is there a way to force hive to read column name from metastore instead of parquet schema ?

Re: Hive select when column name different in parquet and impala

EricL — Thu, 26 Oct 2017 10:55:04 GMT

You might want to try map parquet tables by index, rather than column name:

SET parquet.column.index.access=true;