hbase:acl NotServingRegionException

[#1]

Hi,

I am trying to create anHBase table from HBase shell with my user. My user has not permissions to create tables, so I am executing the following command to grant the permissons:

grant 'root', 'RWCA' 

and I am getting the following ERROR:

org.apache.hadoop.hbase.NotServingRegionException: hbase:acl,,1536838375676.22dbfaf2d9dc840941808287c0c8be67. is not online on node13.agatha.com,16020,1545207998046
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2407)
        at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42010)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:40

9)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

I tried to fix it by repairing the affected regions with metada repair commands like hbase hbck -repair, but command help prompts and says that these commands are not available yet on my hbase version.

I read more about the dissable metadata repair options on HBase 2+ and it seems when restarting hbase master server the consistency tests runs and should repair damaged regions automatically, but it does not happen and I can't grant permissions anymore.

How can I fix it? Thanks

[#2]

Hi @Daniel Hernández

Would you please try the below command.

# /bin/hbase hbck -fixAssignments "hbase:acl"

[#3]

Hi @Sampath Kumar,

Thank you for your answer. As I explain in my question, this is the output I get when trying metadata repair commands like -fixAssignments:

NOTE: Following options are NOT supported as of HBase version 2.0+. UNSUPPORTED Metadata Repair options: (expert features, use with caution!) -fix Try to fix region assignments. This is for backwards compatiblity -fixAssignments Try to fix region assignments. Replaces the old -fix -fixMeta Try to fix meta problems. This assumes HDFS region info is good. -fixHdfsHoles Try to fix region holes in hdfs. -fixHdfsOrphans Try to fix region dirs with no .regioninfo file in hdfs -fixTableOrphans Try to fix table dirs with no .tableinfo file in hdfs (online mode only) -fixHdfsOverlaps Try to fix region overlaps in hdfs. -maxMerge <n> When fixing region overlaps, allow at most <n> regions to merge. (n=5 by default) -sidelineBigOverlaps When fixing region overlaps, allow to sideline big overlaps -maxOverlapsToSideline <n> When fixing region overlaps, allow at most <n> regions to sideline per group. (n=2 by default) -fixSplitParents Try to force offline split parents to be online. -removeParents Try to offline and sideline lingering parents and keep daughter regions. -fixEmptyMetaCells Try to fix hbase:meta entries not referencing any region (empty REGIONINFO_QUALIFIER rows) UNSUPPORTED Metadata Repair shortcuts -repair Shortcut for -fixAssignments -fixMeta -fixHdfsHoles -fixHdfsOrphans -fixHdfsOverlaps -fixVersionFile -sidelineBigOverlaps -fixReferenceFiles-fixHFileLinks -repairHoles Shortcut for -fixAssignments -fixMeta -fixHdfsHoles

I read on the web that on HBase > 2.0 the fixes are run automatically everytime HBase master starts

[#4]

Hi,

As described in

https://jira.apache.org/jira/browse/HBASE-18792

The hbck utility is only provided to HBase < 2.0, and have blocked actions that can corrupt metadata in HBase > 2.0

There is a patch for hbck to work in HBase > 2.0 in this JIRA:

https://jira.apache.org/jira/browse/HBASE-19121

It's not closed and i have not tried to apply the patch, but it could help.