Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

querying a partition table

Solved Go to solution
Highlighted

querying a partition table

New Contributor

Hello Exports

I have created a partition table as below

SET hive.exec.dynamic.partition=true; SET hive.exec.dynamic.partition.mode=nonstrict; CREATE EXTERNAL TABLE `table`( no of clomuns )) PARTITIONED BY (year INT,month INT,day INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS parquet LOCATION 'path';

While querying the tables with on of the partition column, I am getting the following error, for all other regular column it is working fine. Thanks.

0: jdbc:hive2:

//> select * from table where year=2015; Error: java.io.IOException: java.lang.IllegalArgumentException: Column [year] was not found in schema! (state=,code=0)

Can you please let me know what I am doing wrong here.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: querying a partition table

@manichinnari555, I noticed that the table was stored as Parquet. HIVE-11401 is a known bug in Hive filtering on a partition column in Parquet. There is no immediate plan to bring this patch into HDP, but a known workaround is to disable predicate pushdown by setting property hive.optimize.index.filter to false.

3 REPLIES 3

Re: querying a partition table

@manichinnari555, I noticed that the table was stored as Parquet. HIVE-11401 is a known bug in Hive filtering on a partition column in Parquet. There is no immediate plan to bring this patch into HDP, but a known workaround is to disable predicate pushdown by setting property hive.optimize.index.filter to false.

Re: querying a partition table

New Contributor

@Chris Nauroth Many thanks for answering the question and you saved my day.

On the same note, I have another question. I have executed the query setting the property to flalse, do we need to set the property to flase every time we execute the query in production?

Or while creating the table I can set that to false?

Thanks again.

Re: querying a partition table

@manichinnari555, I'm glad to hear this helped. I believe setting at table creation time should be sufficient.