Reply
Highlighted
Explorer
Posts: 40
Registered: ‎01-13-2017

Spark 2.0 : Caused by: MetaException(message:Filtering is supported only on partition keys of type

Hi.

 

SparkSQL 2.1 throws the following warning :

 

 WARN client.Shim_v1_1: Caught Hive MetaException attempting to get partition metadata by filter from Hive. Falling back to fetching all partition metadata, which will degrade performance. Modifying your Hive metastore configuration to set hive.metastore.try.direct.sql to true may resolve this problem.

.....

......

Caused by: MetaException(message:Filtering is supported only on partition keys of type string)

 

I have 2 tables being partition by a data field type INT and the warning only appears for 1 table only.

 

SELECT * FROM table1.... No warning

SELECT * FROM table2.... Warning

 

hive.metastore.try.direct.sql is already set to true.

 

Is it a Bug ? or missing configuration ?

 

thx

Announcements