28732
DISCUSSIONS
101748
MEMBERS
3157
ARTICLES
Created 07-20-2016 01:07 PM
We have thousands of tables and hundreds of partitions for each of those tables.
After enabling Sentry and HDFS ACL sync, it now take 2 full hours before Hive is usable again after restarting.
Metastore logs are filled with log entries that look like this:
Jul 20, 3:00:02.313 PM INFO org.apache.hadoop.hive.metastore.HiveMetaStore 9: get_partitions_by_names : db=**** tbl=*****
Jul 20, 3:00:02.313 PM INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit ugi=hive ip=unknown-ip-addr cmd=get_partitions_by_names : db=**** tbl=****
It looks like it is calling the get_partitions_by_names 10 times per table.
Is there any way to get the HMS to load faster?
/* Joey */