Hbase service check is failing

[#1]

Hbase service check is failing with below error:

ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
	at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2829)
	at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2085)
	at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:706)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:359)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:339)

For usage try 'help "create"'

Took 9.2549 seconds
hbase:025:0> put 'ambarismoketest','row01','family:col01','id620aaa55_date300323'

ERROR: Unknown table ambarismoketest!

For usage try 'help "put"'

Took 0.0754 seconds
hbase:026:0> scan 'ambarismoketest'
ROW  COLUMN+CELL

ERROR: Unknown table ambarismoketest!

For usage try 'help "scan"'

Took 0.0162 seconds
hbase:027:0> exithbase:027:0> 
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2366: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: bad substitution
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2461: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS: bad substitution
SLF4J: Class path contains multiple SLF4J bindings.

HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.4.11.3.2.2.0-1, rUnknown, Mon Sep  4 12:13:14 CEST 2023
Took 0.0043 seconds
hbase:001:0> scan 'ambarismoketest'
ROW  COLUMN+CELL

ERROR: Unknown table ambarismoketest!

For usage try 'help "scan"'

Took 0.5639 seconds
hbase:002:0> 
Looking for id620aaa55_date300323
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2366: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: bad substitution
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2461: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS: bad substitution
SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
hbase:001:0> 
hbase:002:0> #
hbase:003:0> #
hbase:004:0> # Licensed to the Apache Software Foundation (ASF) under one
hbase:005:0> # or more contributor license agreements.  See the NOTICE file
hbase:006:0> # distributed with this work for additional information
hbase:007:0> # regarding copyright ownership.  The ASF licenses this file
hbase:008:0> # to you under the Apache License, Version 2.0 (the
hbase:009:0> # "License"); you may not use this file except in compliance
hbase:010:0> # with the License.  You may obtain a copy of the License at
hbase:011:0> #
hbase:012:0> #   http://www.apache.org/licenses/LICENSE-2.0
hbase:013:0> #
hbase:014:0> # Unless required by applicable law or agreed to in writing,
hbase:015:0> # software distributed under the License is distributed on an
hbase:016:0> # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
hbase:017:0> # KIND, either express or implied.  See the License for the
hbase:018:0> # specific language governing permissions and limitations
hbase:019:0> # under the License.
hbase:020:0> #
hbase:021:0> #
hbase:022:0> disable 'ambarismoketest'

ERROR: Table ambarismoketest does not exist.

For usage try 'help "disable"'

Took 0.7115 seconds
hbase:023:0> drop 'ambarismoketest'

ERROR: Table ambarismoketest does not exist.

For usage try 'help "drop"'

Took 0.0188 seconds
hbase:024:0> create 'ambarismoketest','family'

Command failed after 1 tries

[#2]

Hello,

The message below indicates that the Hbase system tables, "hbase:meta" and "hbase:namespace" are not online or assigned to any Region Server.

If you grep the master logs for the name of the system tables you might be able to see the problem.

If the system table regions are in fact unassigned you can try a full restart if it is a test cluster, you can also use the hbck2 jar to assign regions.

https://github.com/apache/hbase-operator-tools/blob/master/hbase-hbck2/README.md 

ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing

"

[#3]

@MR_KD Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.

[#4 · Accepted solution]

Hi @DianaTorres, I have seen below error in the logs:

ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'ambari-qa' (action=create).

After providing the permissions to ambari-qa in Ranger hbase policy, service checks has been completed successfully.

RCA:
ambari-qa user is the super user and should have sufficient permissions.

Thanks,
MR_KD