Created on 05-10-202111:26 PM - edited on 05-10-202111:33 PM by VidyaSargur
Partition discovery such as "metastore.partition.management.task.frequency" and "partition.retention.period" does not take effect in Hive if metastore.compactor.initiator.on is not turned on by default on HMS. The same property is responsible for activating Hive Metastore background tasks which involves partition discovery as well.
To ensure partition discovery works as expected, do the following:
Go to CM > Hive >Configuration.
Search for Hive Metastore Server Advanced Configuration Snippet (Safety Valve) for hive-site.xml.
Click on + >Add the following:
Name :--> metastore.compactor.initiator.on
Value :--> true
Save the changes and restart.
Ensure the tables have the property discover.partitions is set to True for certain tables.