Support Questions
Find answers, ask questions, and share your expertise

Run Hbase Major Compaction on Particular yarn queue

New Contributor

I would like to run Hbase major compaction on particular yarn queue. so that way the compaction activity will not impact the other resources in the cluster. Do we have option to so. 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Run Hbase Major Compaction on Particular yarn queue

Expert Contributor

Hello @krishpuvi 

 

Thanks for using Cloudera Community. Based on the Post, You wish to run HBase Major Compaction on a particular queue to ensure the Compaction activity doesn't impact other resources in the Cluster. 

 

Please note that HBase Major Compaction isn't using any YARN managed resources & as such, We can't map Major Compaction to be run on any YARN Queue. You can restict MajorCompaction to use lesser bandwidth via [1] & [2], which explains the "PressureAwareCompactionThroughputController" with Min & Max Speed along with the Parameter to disable any Speed Limit via "NoLimitThroughputController". Additionally, You can disable Compaction & run Compaction manually during Off-Business-Hours at Table/ColumnFamiliy Level as well. 

 

- Smarak

 

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/configuring-hbase/topics/hbase-limit-the-spee...

[2] https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/throttle/PressureAwareCompa...

 

View solution in original post

1 REPLY 1

Re: Run Hbase Major Compaction on Particular yarn queue

Expert Contributor

Hello @krishpuvi 

 

Thanks for using Cloudera Community. Based on the Post, You wish to run HBase Major Compaction on a particular queue to ensure the Compaction activity doesn't impact other resources in the Cluster. 

 

Please note that HBase Major Compaction isn't using any YARN managed resources & as such, We can't map Major Compaction to be run on any YARN Queue. You can restict MajorCompaction to use lesser bandwidth via [1] & [2], which explains the "PressureAwareCompactionThroughputController" with Min & Max Speed along with the Parameter to disable any Speed Limit via "NoLimitThroughputController". Additionally, You can disable Compaction & run Compaction manually during Off-Business-Hours at Table/ColumnFamiliy Level as well. 

 

- Smarak

 

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/configuring-hbase/topics/hbase-limit-the-spee...

[2] https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/throttle/PressureAwareCompa...

 

View solution in original post