Support Questions

Find answers, ask questions, and share your expertise

Hbase service check is failing

avatar
Explorer

Hbase service check is failing with below error:

ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
	at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2829)
	at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2085)
	at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:706)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:359)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:339)

For usage try 'help "create"'

Took 9.2549 seconds
hbase:025:0> put 'ambarismoketest','row01','family:col01','id620aaa55_date300323'

ERROR: Unknown table ambarismoketest!

For usage try 'help "put"'

Took 0.0754 seconds
hbase:026:0> scan 'ambarismoketest'
ROW  COLUMN+CELL

ERROR: Unknown table ambarismoketest!

For usage try 'help "scan"'

Took 0.0162 seconds
hbase:027:0> exithbase:027:0> 
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2366: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: bad substitution
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2461: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS: bad substitution
SLF4J: Class path contains multiple SLF4J bindings.

HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.4.11.3.2.2.0-1, rUnknown, Mon Sep  4 12:13:14 CEST 2023
Took 0.0043 seconds
hbase:001:0> scan 'ambarismoketest'
ROW  COLUMN+CELL

ERROR: Unknown table ambarismoketest!

For usage try 'help "scan"'

Took 0.5639 seconds
hbase:002:0> 
Looking for id620aaa55_date300323
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2366: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: bad substitution
/usr/odp/3.2.2.0-1/hadoop/libexec/hadoop-functions.sh: line 2461: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS: bad substitution
SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
hbase:001:0> 
hbase:002:0> #
hbase:003:0> #
hbase:004:0> # Licensed to the Apache Software Foundation (ASF) under one
hbase:005:0> # or more contributor license agreements.  See the NOTICE file
hbase:006:0> # distributed with this work for additional information
hbase:007:0> # regarding copyright ownership.  The ASF licenses this file
hbase:008:0> # to you under the Apache License, Version 2.0 (the
hbase:009:0> # "License"); you may not use this file except in compliance
hbase:010:0> # with the License.  You may obtain a copy of the License at
hbase:011:0> #
hbase:012:0> #   http://www.apache.org/licenses/LICENSE-2.0
hbase:013:0> #
hbase:014:0> # Unless required by applicable law or agreed to in writing,
hbase:015:0> # software distributed under the License is distributed on an
hbase:016:0> # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
hbase:017:0> # KIND, either express or implied.  See the License for the
hbase:018:0> # specific language governing permissions and limitations
hbase:019:0> # under the License.
hbase:020:0> #
hbase:021:0> #
hbase:022:0> disable 'ambarismoketest'

ERROR: Table ambarismoketest does not exist.

For usage try 'help "disable"'

Took 0.7115 seconds
hbase:023:0> drop 'ambarismoketest'

ERROR: Table ambarismoketest does not exist.

For usage try 'help "drop"'

Took 0.0188 seconds
hbase:024:0> create 'ambarismoketest','family'

Command failed after 1 tries

 

1 ACCEPTED SOLUTION

avatar
Explorer

Hi @DianaTorres, I have seen below error in the logs:

 

ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'ambari-qa' (action=create).

After providing the permissions to ambari-qa in Ranger hbase policy, service checks has been completed successfully.

RCA:
ambari-qa user is the super user and should have sufficient permissions.

Thanks,
MR_KD

View solution in original post

3 REPLIES 3

avatar
Cloudera Employee

Hello,

The message below indicates that the Hbase system tables, "hbase:meta" and "hbase:namespace" are not online or assigned to any Region Server.

If you grep the master logs for the name of the system tables you might be able to see the problem.

If the system table regions are in fact unassigned you can try a full restart if it is a test cluster, you can also use the hbck2 jar to assign regions.

https://github.com/apache/hbase-operator-tools/blob/master/hbase-hbck2/README.md 

ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing

"

avatar
Community Manager

@MR_KD Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Explorer

Hi @DianaTorres, I have seen below error in the logs:

 

ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'ambari-qa' (action=create).

After providing the permissions to ambari-qa in Ranger hbase policy, service checks has been completed successfully.

RCA:
ambari-qa user is the super user and should have sufficient permissions.

Thanks,
MR_KD