04-13-2017 02:06 PM
SparkSQL 2.1 throws the following warning :
WARN client.Shim_v1_1: Caught Hive MetaException attempting to get partition metadata by filter from Hive. Falling back to fetching all partition metadata, which will degrade performance. Modifying your Hive metastore configuration to set hive.metastore.try.direct.sql to true may resolve this problem.
Caused by: MetaException(message:Filtering is supported only on partition keys of type string)
I have 2 tables being partition by a data field type INT and the warning only appears for 1 table only.
SELECT * FROM table1.... No warning
SELECT * FROM table2.... Warning
hive.metastore.try.direct.sql is already set to true.
Is it a Bug ? or missing configuration ?