Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

Hive Metastore Takes Two Hours to Start

avatar
Contributor

We have thousands of tables and hundreds of partitions for each of those tables.

After enabling Sentry and HDFS ACL sync, it now take 2 full hours before Hive is usable again after restarting.

 

Metastore logs are filled with log entries that look like this:

Jul 20, 3:00:02.313 PM INFO org.apache.hadoop.hive.metastore.HiveMetaStore 9: get_partitions_by_names : db=**** tbl=*****
Jul 20, 3:00:02.313 PM INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit ugi=hive ip=unknown-ip-addr cmd=get_partitions_by_names : db=**** tbl=****

 

It looks like it is calling the get_partitions_by_names 10 times per table.

 

Is there any way to get the HMS to load faster?

 

/* Joey */

Who agreed with this topic