Reply
Explorer
Posts: 25
Registered: ‎01-30-2014
Accepted Solution

CDH4.5 enable ExploringCompaction

[ Edited ]

We're moving from CDH4.0.0 to CDH4.5.0 and I would like to use newer ExploringCompaction policy on my HBase tables, since we're using bulkload for our ingest and we're running into problems with de default policy.

 

From [1] and [2] I read that there should be an option to enable the new policy by setting the "hbase.hstore.useExploringCompation". 

 

My questions are:

* Is this available in CDH4.5.0 running with hbase 0.94.6? 

* Do i set this in the safety valvle for the regionserver settings?

* Can I verify this is the current policy running on major compactions?

 

[1] http://blog.cloudera.com/blog/2013/09/how-to-use-hbase-bulk-loading-and-why/

[2] htps://issues.apache.org/jira/browse/HBASE-8283

Cloudera Employee J-D
Cloudera Employee
Posts: 60
Registered: ‎07-08-2013

Re: CDH4.5 enable ExploringCompaction

* Is this available in CDH4.5.0 running with hbase 0.94.6?

 

It is available in 0.94.6-cdh4.5.0 yes.

 

* Do i set this in the safety valvle for the regionserver settings?

 

Yes.

 

* Can I verify this is the current policy running on major compactions?

 

Major compactions compact all the files, so there isn't a policy in use here. Best I can tell you can't really see if the policy is in effect when file selection is done for minor compactions. One thing you can do is look at the list of configurations on a running region server, go to one of your region server's web UI and put /conf after the port then look for hbase.hstore.useExploringCompation. It won't be listed in 4.0.0 but in 4.5.0 it should be there and set to true if you set useExploringCompation correctly.

Posts: 1,896
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: CDH4.5 enable ExploringCompation

> * Is this available in CDH4.5.0 running with hbase 0.94.6?

Yes. See https://github.com/cloudera/hbase/blob/cdh4.5.0-release/src/main/java/org/apache/hadoop/hbase/region...

> * Do i set this in the safety valvle for the regionserver settings?

Yes, this goes into the Advanced properties (safety valve for
hbase-site.xml) of the Region Servers, as thats where compactions of
store files are performed.

> * Can I verify this is the current policy running on major compactions?

There's no direct way AFAICT, but setting the property should suffice
in enabling that sub-routine, per the logic shown above.

Posts: 1,896
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: CDH4.5 enable ExploringCompation

Ah I didn't notice JD's post when replying by email, my bad.
Announcements

Our community is getting a little larger. And a lot better.


Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.