Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

In HDP 2.x we had hbck tool which was used to identify and fix inconsistencies in hbase tables. However with HBase 2.x and HDP 3.x together this tool is deprecated and its strongly advised to not run it on production clusters as it may lead to data corruption.

HBase 2.0 introduces a completely revamped HBase Assignment Manager with Proc V2 framework (State machine). In HDP-2.x, we maintained HBASE_REGION_STATE and TABLE_STATE in zookeeper and Hmaster in-memory ( possibility for inconsistencies) but with HDP-3 it uses MasterProcWAL procedure which is stored in HDFS.

These set of procedures are now scheduled and executed by HMaster.

Given the changes on internals of new AssignmentManager , the old hbck fix options no longer work, but it can still provide a report about tables/regions states. To fix regions assignment problems that may still occur, a new HBCK 2 has been designed, and shipped independently from hbase build artifacts (and hence not part of HDP 3.x distribution ), as a plan to have it able to evolve on its own, with new fix options for previously unforeseen issues added and integrated into the tool without the need of whole new hbase release.

This HBCK2 tool as of now is not as intuitive and useful as old hbck but it helps fix issues related to Proc v2 (introduced in HBase 2.0) and other issues such as region in transition.


Note: HBCK2 tool requires minimum HBase 2.0.3 which comes starting HDP 3.1


To build hbase-hbck2:
# cd <work directory>
# git clone https://github.com/apache/hbase-operator-tools.git
# cd hbase-operator-tools/
# mvn clean install -DskipTests
# cd hbase-hbck2/target
# Download hbase-hbck2-1.0.0-SNAPSHOT.jar


Running HBCK2


Once HBCK2 has been uploaded to the target cluster, it can be executed by specifying its jar path in the "-j" option of "hbck" command as below:


# su - hbase
$ hbase hbck -j <jar path>/hbase-hbck2-1.0.0-SNAPSHOT.jar <COMMAD>

You may encounter following error running in HDP 3.0


Exception in thread "main" java.lang.UnsupportedOperationException: Requires 2.0.3 at least.

Use -s option, instead.
$ hbase hbck -j <jar path>/hbase-hbck2-1.0.0-SNAPSHOT.jar -s <COMMAND>


Reference:


Special mention and thanks to Karthik Palanisamy for critical inputs here.




224 Views
0 Kudos
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎07-02-2019 08:10 AM
Updated by:
 
Contributors
Top Kudoed Authors