Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

After restart HBase region server role instance,becoming Down

After restart HBase region server role instance,becoming Down

I have added block-pinning property to HBase region server's hbsae-site.xml file.

 

The property is

 

<property>

<name>dfs.datanode.block-pinning.enabled</name>

<value>true</value>

</property>

 

To take effect I have restarted the RegionServer role instance. Then it became Down.

 

The log is showing following error

 

3:44:58.820 PMERROROpenRegionHandler
Failed open of region=hbase:meta,,1.1588230740, starting to roll back the global memstore size.
java.io.InterruptedIOException
	at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:108)
	at org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
	at org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
	at org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1022)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6645)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6599)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6570)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6526)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6477)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:362)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
	at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
	at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
	... 14 more
3:44:58.821 PMINFOZkOpenRegionCoordination
Opening of region {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in ZK, expecting version 17
3:44:58.961 PMINFORpcServer
Stopping server on 60020
3:44:58.961 PMINFORpcServer
RpcServer.listener,port=60020: stopping
3:44:58.961 PMINFORpcServer
RpcServer.responder: stopped
3:44:58.961 PMINFORpcServer
RpcServer.responder: stopping
3:44:59.065 PMINFOClientCnxn
EventThread shut down
3:44:59.065 PMINFOZooKeeper
Session: 0x165fac7ed380f27 closed
3:44:59.065 PMINFOHRegionServer
stopping server chinni,60020,1537611276938; zookeeper connection closed.
3:44:59.065 PMINFOHRegionServer
regionserver/chinni/127.0.0.1:60020 exiting
3:44:59.066 PMERRORHRegionServerCommandLine
Region server exiting
java.lang.RuntimeException: HRegionServer Aborted
	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2752)
3:44:59.067 PMINFOShutdownHook
Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@70605759
3:44:59.067 PMINFOShutdownHook
Starting fs shutdown hook thread.
3:44:59.071 PMINFOShutdownHook
Shutdown hook finished.

 

 

Any fixes?

 

Thanks and regards 
solomonchinni