question querying a partition table in Archives of Support Questions (Read Only)

querying a partition table

venkatibrahmam_ — Fri, 17 Jun 2016 22:46:26 GMT

Hello Exports

I have created a partition table as below

SET hive.exec.dynamic.partition=true; SET hive.exec.dynamic.partition.mode=nonstrict; CREATE EXTERNAL TABLE `table`( no of clomuns )) PARTITIONED BY (year INT,month INT,day INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS parquet LOCATION 'path';

While querying the tables with on of the partition column, I am getting the following error, for all other regular column it is working fine. Thanks.

0: jdbc:hive2:

//> select * from table where year=2015; Error: java.io.IOException: java.lang.IllegalArgumentException: Column [year] was not found in schema! (state=,code=0)

Can you please let me know what I am doing wrong here.

Re: querying a partition table

cnauroth — Sat, 18 Jun 2016 01:07:30 GMT

@manichinnari555, I noticed that the table was stored as Parquet. HIVE-11401 is a known bug in Hive filtering on a partition column in Parquet. There is no immediate plan to bring this patch into HDP, but a known workaround is to disable predicate pushdown by setting property hive.optimize.index.filter to false.

Re: querying a partition table

venkatibrahmam_ — Sat, 18 Jun 2016 02:02:28 GMT

@Chris Nauroth Many thanks for answering the question and you saved my day.

On the same note, I have another question. I have executed the query setting the property to flalse, do we need to set the property to flase every time we execute the query in production?

Or while creating the table I can set that to false?

Thanks again.

Re: querying a partition table

cnauroth — Sat, 18 Jun 2016 02:24:59 GMT

@manichinnari555, I'm glad to hear this helped. I believe setting at table creation time should be sufficient.