Support Questions
Find answers, ask questions, and share your expertise

HBase Cluster

Hello All,


In my cluster enviroment Master service is shutting down automatically.Here are logs


2014-07-17 09:09:47,232 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1967360323- file=/hbase/


Any hints ?


Re: HBase Cluster

Master Guru
Somehow your file of /hbase/ has lost its only block, turning corrupt. I'd recommend inspecting your HDFS health (via 'hdfs fsck /', and also by checking if all DNs are alive and in good health).

In terms of recovery, if you are unable to recover the block back from the DataNodes, then you can proceed with stopping all HBase services, deleting the /hbase/ file, and then restarting the HBase service.

P.s. Another easier fix would be to use HBCK, but the required fixing feature (for cluster IDs) is incomplete yet: