Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
avatar
Expert Contributor

Issue

Partition discovery such as "metastore.partition.management.task.frequency" and "partition.retention.period" does not take effect in Hive if metastore.compactor.initiator.on is not turned on by default on HMS. The same property is responsible for activating Hive Metastore background tasks which involves partition discovery as well.

 

Resolution

To ensure partition discovery works as expected, do the following:

 

  1. Go to CM > Hive >Configuration.
  2. Search for Hive Metastore Server Advanced Configuration Snippet (Safety Valve) for hive-site.xml.
  3. Click on + >Add the following:
    Name :--> metastore.compactor.initiator.on 
    Value :--> true
  4. Save the changes and restart.
  5. Ensure the tables have the property discover.partitions is set to True for certain tables.
1,007 Views
0 Kudos