Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

FATAL [main-EventThread] regionserver.HRegionServer: ABORTING region server

FATAL [main-EventThread] regionserver.HRegionServer: ABORTING region server

New Contributor

Hello team,

I have a problem with region server aborted.

Part of logs is attached:

2017-02-21 14:32:41,017 FATAL [main-EventThread] regionserver.HRegionServer: ABORTING region server serv01,16020,1486954210496: regionserver:16020-0x35a173e7b40003f, quorum=serv02.localdomain:2181,serv03.localdomain:2181,serv01.localdomain: 2181, baseZNode=/hbase-unsecure regionserver:16020-0x35a173e7b40003f received expired from ZooKeeper, aborting org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:610) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:521) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:534) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2017-02-21 14:32:43,438 FATAL [regionserver/serv01/10.0.7.1:16020] regionserver.HRegionServer: ABORTING region server server01,16020,1486954210496: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing serv01,1 6020,1486954210496 as dead server at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:413) at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:318) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:325) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionSe rverStatusProtos.java:8617) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) at java.lang.Thread.run(Thread.java:745) org.apache.hadoop.hbase.YouAreDeadException: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing serv01,16020,1486954210496 as dead server at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:413) at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:318) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:325) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionSe rverStatusProtos.java:8617) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) at java.lang.Thread.run(Thread.java:745) 2017-02-21 14:32:43,879 INFO [regionserver/serv01/10.0.7.1:16020] regionserver.HRegionServer: STOPPED: org.apache.hadoop.hb ase.YouAreDeadException: Server REPORT rejected; currently processing serv01,16020,1486954210496 as dead server at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:413) at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:318) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:325) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionSe rverStatusProtos.java:8617) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) at java.lang.Thread.run(Thread.java:745) 2017-02-21 14:33:05,801 ERROR [regionserver/serv01/10.0.7.1:16020] zookeeper.ZooKeeperWatcher: regionserver:16020-0x35a173e7 b40003f, quorum=serv02.localdomain:2181,serv03.localdomain:2181,serv01.localdomain:2181, baseZNode=/hbase-unsecur e Received unexpected KeeperException, re-throwing exception org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase-unsecure/replication/rs/serv01,16020,1486954210496 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getChildren(RecoverableZooKeeper.java:295) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:454) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchThem(ZKUtil.java:482) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenBFSAndWatchThem(ZKUtil.java:1461) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeRecursivelyMultiOrSequential(ZKUtil.java:1383) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeRecursively(ZKUtil.java:1265) at org.apache.hadoop.hbase.replication.ReplicationQueuesZKImpl.removeAllQueues(ReplicationQueuesZKImpl.java:187) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:292) at org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:180) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:172) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2137) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1071) at java.lang.Thread.run(Thread.java:745)

Please help.

Don't have an account?
Coming from Hortonworks? Activate your account here