Reply
Highlighted
New Contributor
Posts: 6
Registered: ‎03-18-2014

HMaster not starting -- Unexpected state

Hi

 

 

While srating HMaster i get the following FATAL error and the HMaster crashes.

 

Here are few which may have caused it

 

1. I disabled and deleted all htables

2. I changed the replication from 3 to 1 for HDFS

3.Restarted HMaster which started giving the following error 

 

 

Unexpected state : canonical_message_customers,,1396004880469.20c78a1262e7979063a92e73de119db1. state=PENDING_OPEN, ts=1398406467556, server=host.com,60020,1398255607638 .. Cannot transit it to OFFLINE.
java.lang.IllegalStateException: Unexpected state : canonical_message_customers,,1396004880469.20c78a1262e7979063a92e73de119db1. state=PENDING_OPEN, ts=1398406467556, server=slc06cab.oracle.com,60020,1398255607638 .. Cannot transit it to OFFLINE.
at org.apache.hadoop.hbase.master.AssignmentManager.setOfflineInZooKeeper(AssignmentManager.java:1831)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1661)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1426)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1398)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1393)
at org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:105)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)

 

Cloudera Employee
Posts: 35
Registered: ‎09-04-2013

Re: HMaster not starting -- Unexpected state

Can you try to shutdown region servers as well, then clean up the hbase
zookeeper node, then start the cluster again?