Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

Spark 2.0 : Caused by: MetaException(message:Filtering is supported only on partition keys of type

avatar

Hi.

 

SparkSQL 2.1 throws the following warning :

 

 WARN client.Shim_v1_1: Caught Hive MetaException attempting to get partition metadata by filter from Hive. Falling back to fetching all partition metadata, which will degrade performance. Modifying your Hive metastore configuration to set hive.metastore.try.direct.sql to true may resolve this problem.

.....

......

Caused by: MetaException(message:Filtering is supported only on partition keys of type string)

 

I have 2 tables being partition by a data field type INT and the warning only appears for 1 table only.

 

SELECT * FROM table1.... No warning

SELECT * FROM table2.... Warning

 

hive.metastore.try.direct.sql is already set to true.

 

Is it a Bug ? or missing configuration ?

 

thx

Who agreed with this topic