question Re: PARQUET_FALLBACK_SCHEMA_RESOLUTION in Archives of Support Questions (Read Only)

PARQUET_FALLBACK_SCHEMA_RESOLUTION

medloh — Tue, 21 Apr 2026 13:45:11 GMT

Is it possible to make Impala default to 'name' instead of 'position' so I don't have to do this every Hue session?

set PARQUET_FALLBACK_SCHEMA_RESOLUTION=name;

My parquet files don't always have the same set of columns, so I have to lookup columns by name.

Re: PARQUET_FALLBACK_SCHEMA_RESOLUTION

tianyiwang — Wed, 29 Nov 2017 00:21:57 GMT

You can change the default query options via impalad command line options:

-default_query_options='PARQUET_FALLBACK_SCHEMA_RESOLUTION=name'

The same can be done in Cloudera Manager: https://www.cloudera.com/documentation/enterprise/5-12-x/topics/impala_config_options.html

Re: PARQUET_FALLBACK_SCHEMA_RESOLUTION

medloh — Wed, 29 Nov 2017 00:32:42 GMT

Thanks, I was able to add it under config--Impala Daemon, Impala Daemon Query Options Advanced Configuration Snippet (Safety Valve), and then add PARQUET_FALLBACK_SCHEMA_RESOLUTION=name. Seems to work.

Do you know if there are any significant consequences to changing this? Seems like name would be a better default to go with if performance was about the same.

Re: PARQUET_FALLBACK_SCHEMA_RESOLUTION

alex.behm — Wed, 29 Nov 2017 01:00:06 GMT

It's generally safe to use name-based resolution by default. Performance should be about the same. I agree name-based resolution may be a better choice because it's more intuitive.

Index vs. name based resolution have different tradeoffs in terms of what schema-evolution operations are allowed. For example with index-based resolution you can safely rename a column in your table schema. With name based resolution you can safely add/drop columns in the middle of your table schema, whereas with index-based resolution you can generally only add new columns at the end. So it's really all about tradeoffs.