Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH4.5 enable ExploringCompaction

SOLVED Go to solution

CDH4.5 enable ExploringCompaction

Contributor

We're moving from CDH4.0.0 to CDH4.5.0 and I would like to use newer ExploringCompaction policy on my HBase tables, since we're using bulkload for our ingest and we're running into problems with de default policy.

 

From [1] and [2] I read that there should be an option to enable the new policy by setting the "hbase.hstore.useExploringCompation". 

 

My questions are:

* Is this available in CDH4.5.0 running with hbase 0.94.6? 

* Do i set this in the safety valvle for the regionserver settings?

* Can I verify this is the current policy running on major compactions?

 

[1] http://blog.cloudera.com/blog/2013/09/how-to-use-hbase-bulk-loading-and-why/

[2] htps://issues.apache.org/jira/browse/HBASE-8283

1 ACCEPTED SOLUTION

Accepted Solutions

Re: CDH4.5 enable ExploringCompaction

Rising Star

* Is this available in CDH4.5.0 running with hbase 0.94.6?

 

It is available in 0.94.6-cdh4.5.0 yes.

 

* Do i set this in the safety valvle for the regionserver settings?

 

Yes.

 

* Can I verify this is the current policy running on major compactions?

 

Major compactions compact all the files, so there isn't a policy in use here. Best I can tell you can't really see if the policy is in effect when file selection is done for minor compactions. One thing you can do is look at the list of configurations on a running region server, go to one of your region server's web UI and put /conf after the port then look for hbase.hstore.useExploringCompation. It won't be listed in 4.0.0 but in 4.5.0 it should be there and set to true if you set useExploringCompation correctly.

3 REPLIES 3

Re: CDH4.5 enable ExploringCompaction

Rising Star

* Is this available in CDH4.5.0 running with hbase 0.94.6?

 

It is available in 0.94.6-cdh4.5.0 yes.

 

* Do i set this in the safety valvle for the regionserver settings?

 

Yes.

 

* Can I verify this is the current policy running on major compactions?

 

Major compactions compact all the files, so there isn't a policy in use here. Best I can tell you can't really see if the policy is in effect when file selection is done for minor compactions. One thing you can do is look at the list of configurations on a running region server, go to one of your region server's web UI and put /conf after the port then look for hbase.hstore.useExploringCompation. It won't be listed in 4.0.0 but in 4.5.0 it should be there and set to true if you set useExploringCompation correctly.

Re: CDH4.5 enable ExploringCompation

Master Guru
> * Is this available in CDH4.5.0 running with hbase 0.94.6?

Yes. See https://github.com/cloudera/hbase/blob/cdh4.5.0-release/src/main/java/org/apache/hadoop/hbase/region...

> * Do i set this in the safety valvle for the regionserver settings?

Yes, this goes into the Advanced properties (safety valve for
hbase-site.xml) of the Region Servers, as thats where compactions of
store files are performed.

> * Can I verify this is the current policy running on major compactions?

There's no direct way AFAICT, but setting the property should suffice
in enabling that sub-routine, per the logic shown above.

Highlighted

Re: CDH4.5 enable ExploringCompation

Master Guru
Ah I didn't notice JD's post when replying by email, my bad.