Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Compaction worker threads checking the same partitions (HDInsights 4.0)

Hive Compaction worker threads checking the same partitions (HDInsights 4.0)

New Contributor

I'm working on a Hive Transactional table (on an Azure HDInsights 4.0 cluster) and pushing data via Hive Streaming. After the delta files are being created, I'm expecting a compaction to happen after an interval of "hive.compactor.check.interval", but to no avail.

 

After checking the hive metastore logs, I discovered that the compaction threads (set to 5) are checking the same set of partitions everytime. I have around 30 partitions for the table, and only the first 5 are being picked to check if compaction is required.

 

Any help with this?

Don't have an account?
Coming from Hortonworks? Activate your account here