HBASE Master failed to restart (related to zookeeper?)

gagabonar — Fri, 16 Sep 2022 09:07:06 GMT

I've just completed with CM and CDH installation and trying to add HBase service to the Master node. This is not production environment no data has been loaded yet; pretty much a clean install.

After adding Hbase service the first to Master Node, I got a bad health alert saying that my disk space is not sufficient. So, what I tried to do was to move hbase log directory to a different location using symlink. I realized that when doing this I forgot to stop the Hbase service. After I'm done with the symlink, I restart the Hbase service and fail. I got the error below.

Question, the Hbase service configuration at CM don't have option to move or update the log directory. What is your suggestion to do this in the future? thanks in advance.

Master exiting
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
	at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2775)
	at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:184)
	at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:134)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
	at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2789)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:489)
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:468)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1233)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1211)
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:174)
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:167)
	at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:472)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2770)
	... 5 more

Re: HBASE Master failed to restart (related to zookeeper?)

GautamG — Mon, 08 Sep 2014 06:14:17 GMT

The error is originating from ZooKeeper. Which directory did you move:
/var/log or /var/log/hbase? Did you try restarting the entire cluster?

Re: HBASE Master failed to restart (related to zookeeper?)

gagabonar — Mon, 08 Sep 2014 07:35:40 GMT

/var/log/hbase

I've restarted the entire cluster. Now it's up again.

But the problem still remains that I'm getting bad health alert because of /var/log disk space is insufficient. I want to change the log directory for hbase to my /apps/ext/var/log directory. But how can I do that?

There are no log storage setting for Hbase in CM.

Thanks for the reply.

Re: HBASE Master failed to restart (related to zookeeper?)

gagabonar — Tue, 09 Sep 2014 03:15:08 GMT

Nevermind, the log storage setting for HBase is available on CM. I didn't realilze those arrows next to category label can be expanded. And there will be 'Log' category which I can update the value.

question Re: HBASE Master failed to restart (related to zookeeper?) in Archives of Support Questions (Read Only)

HBASE Master failed to restart (related to zookeeper?)

Re: HBASE Master failed to restart (related to zookeeper?)

Re: HBASE Master failed to restart (related to zookeeper?)

Re: HBASE Master failed to restart (related to zookeeper?)