question Parquet index page for Impala in Archives of Support Questions (Read Only)

Parquet index page for Impala

fil — Fri, 16 Sep 2022 09:33:03 GMT

Hi dear experts!

could anybody know does parquet index page available for Impala(https://github.com/Parquet/parquet-format)? if so, where i can find more information about that?

thank you in advance!

Re: Parquet index page for Impala

alex.behm — Wed, 08 Jul 2015 15:21:13 GMT

Hi fil,

I'm afraid Impala currently does not take advantage of index pages when reading/writing data.

Index pages are part of the Parquet spec, so in theory, you shoud be able to write Parquet files with index pages (via some other tool) and have them be readable by Impala - but Impala will ignore the index pages.

We do plan on taking advantage of index pages and min/max/etc values, but we do not have a concrete target date for that feature yet.

Alex

Re: Parquet index page for Impala

fil — Wed, 08 Jul 2015 17:44:33 GMT

thanks for your comment!

Re: Parquet index page for Impala

Dautkhanov — Mon, 21 Nov 2016 06:46:01 GMT

https://github.com/Parquet/parquet-format/blob/f7ab552f569df63bdb59f751d0dd36e826682739/src/thrift/parquet.thrift#L338

Index pages are declared in Parquet format, but not actually implemented.

See code above.