Reply
Highlighted
Explorer
Posts: 28
Registered: ‎04-15-2018

After restart HBase region server role instance,becoming Down

I have added block-pinning property to HBase region server's hbsae-site.xml file.

 

The property is

 

<property>

<name>dfs.datanode.block-pinning.enabled</name>

<value>true</value>

</property>

 

To take effect I have restarted the RegionServer role instance. Then it became Down.

 

The log is showing following error

 

3:44:58.820 PMERROROpenRegionHandler
Failed open of region=hbase:meta,,1.1588230740, starting to roll back the global memstore size.
java.io.InterruptedIOException
	at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:108)
	at org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
	at org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
	at org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1022)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6645)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6599)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6570)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6526)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6477)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:362)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
	at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
	at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
	... 14 more
3:44:58.821 PMINFOZkOpenRegionCoordination
Opening of region {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in ZK, expecting version 17
3:44:58.961 PMINFORpcServer
Stopping server on 60020
3:44:58.961 PMINFORpcServer
RpcServer.listener,port=60020: stopping
3:44:58.961 PMINFORpcServer
RpcServer.responder: stopped
3:44:58.961 PMINFORpcServer
RpcServer.responder: stopping
3:44:59.065 PMINFOClientCnxn
EventThread shut down
3:44:59.065 PMINFOZooKeeper
Session: 0x165fac7ed380f27 closed
3:44:59.065 PMINFOHRegionServer
stopping server chinni,60020,1537611276938; zookeeper connection closed.
3:44:59.065 PMINFOHRegionServer
regionserver/chinni/127.0.0.1:60020 exiting
3:44:59.066 PMERRORHRegionServerCommandLine
Region server exiting
java.lang.RuntimeException: HRegionServer Aborted
	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2752)
3:44:59.067 PMINFOShutdownHook
Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@70605759
3:44:59.067 PMINFOShutdownHook
Starting fs shutdown hook thread.
3:44:59.071 PMINFOShutdownHook
Shutdown hook finished.

 

 

Any fixes?

 

Thanks and regards 
solomonchinni

 

Announcements