CDH4.5 enable ExploringCompaction

[#1]

We're moving from CDH4.0.0 to CDH4.5.0 and I would like to use newer ExploringCompaction policy on my HBase tables, since we're using bulkload for our ingest and we're running into problems with de default policy.

From [1] and [2] I read that there should be an option to enable the new policy by setting the "hbase.hstore.useExploringCompation". 

My questions are:

* Is this available in CDH4.5.0 running with hbase 0.94.6? 

* Do i set this in the safety valvle for the regionserver settings?

* Can I verify this is the current policy running on major compactions?

[1] http://blog.cloudera.com/blog/2013/09/how-to-use-hbase-bulk-loading-and-why/

[2] htps://issues.apache.org/jira/browse/HBASE-8283

[#2 · Accepted solution]

* Is this available in CDH4.5.0 running with hbase 0.94.6?

It is available in 0.94.6-cdh4.5.0 yes.

* Do i set this in the safety valvle for the regionserver settings?

Yes.

* Can I verify this is the current policy running on major compactions?

Major compactions compact all the files, so there isn't a policy in use here. Best I can tell you can't really see if the policy is in effect when file selection is done for minor compactions. One thing you can do is look at the list of configurations on a running region server, go to one of your region server's web UI and put /conf after the port then look for hbase.hstore.useExploringCompation. It won't be listed in 4.0.0 but in 4.5.0 it should be there and set to true if you set useExploringCompation correctly.

[#3]

> * Is this available in CDH4.5.0 running with hbase 0.94.6?

Yes. See https://github.com/cloudera/hbase/blob/cdh4.5.0-release/src/main/java/org/apache/hadoop/hbase/region...

> * Do i set this in the safety valvle for the regionserver settings?

Yes, this goes into the Advanced properties (safety valve for
hbase-site.xml) of the Region Servers, as thats where compactions of
store files are performed.

> * Can I verify this is the current policy running on major compactions?

There's no direct way AFAICT, but setting the property should suffice
in enabling that sub-routine, per the logic shown above.

[#4]

Ah I didn't notice JD's post when replying by email, my bad.