Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

querying a partition table

Solved Go to solution

querying a partition table

Hello Exports

I have created a partition table as below

SET hive.exec.dynamic.partition=true; SET hive.exec.dynamic.partition.mode=nonstrict; CREATE EXTERNAL TABLE `table`( no of clomuns )) PARTITIONED BY (year INT,month INT,day INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS parquet LOCATION 'path';

While querying the tables with on of the partition column, I am getting the following error, for all other regular column it is working fine. Thanks.

0: jdbc:hive2:

//> select * from table where year=2015; Error: java.io.IOException: java.lang.IllegalArgumentException: Column [year] was not found in schema! (state=,code=0)

Can you please let me know what I am doing wrong here.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: querying a partition table

@manichinnari555, I noticed that the table was stored as Parquet. HIVE-11401 is a known bug in Hive filtering on a partition column in Parquet. There is no immediate plan to bring this patch into HDP, but a known workaround is to disable predicate pushdown by setting property hive.optimize.index.filter to false.

View solution in original post

3 REPLIES 3
Highlighted

Re: querying a partition table

@manichinnari555, I noticed that the table was stored as Parquet. HIVE-11401 is a known bug in Hive filtering on a partition column in Parquet. There is no immediate plan to bring this patch into HDP, but a known workaround is to disable predicate pushdown by setting property hive.optimize.index.filter to false.

View solution in original post

Highlighted

Re: querying a partition table

@Chris Nauroth Many thanks for answering the question and you saved my day.

On the same note, I have another question. I have executed the query setting the property to flalse, do we need to set the property to flase every time we execute the query in production?

Or while creating the table I can set that to false?

Thanks again.

Highlighted

Re: querying a partition table

@manichinnari555, I'm glad to hear this helped. I believe setting at table creation time should be sufficient.
Don't have an account?
Coming from Hortonworks? Activate your account here