Support Questions

Find answers, ask questions, and share your expertise

Region server exiting java.lang.RuntimeException: HRegionServer Aborted

avatar
Explorer

10:07:02.656 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer
STOPPED: Unexpected exception handling getData
10:07:02.665 AM INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker
Sending interrupt to stop the worker thread
10:07:02.677 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer
Stopping infoServer
10:07:02.697 AM INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker
SplitLogWorker interrupted. Exiting.
10:07:02.698 AM INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker
SplitLogWorker big-bdls02.audi.local,60020,1459747837953 exiting
10:07:02.728 AM INFO org.mortbay.log
Stopped SelectChannelConnector@0.0.0.0:60030
10:07:02.879 AM INFO org.apache.hadoop.hbase.regionserver.HeapMemoryManager
Stoping HeapMemoryTuner chore.
10:07:02.879 AM INFO org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager
Stopping RegionServerSnapshotManager abruptly.
10:07:02.879 AM INFO org.apache.hadoop.hbase.procedure.flush.RegionServerFlushTableProcedureManager
Stopping region server flush procedure manager abruptly.
10:07:02.880 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer
aborting server big-bdls02.audi.local,60020,1459747837953
10:07:02.914 AM INFO org.apache.hadoop.hbase.regionserver.HeapMemoryManager$HeapMemoryTunerChore
big-bdls02.audi.local,60020,1459747837953-HeapMemoryTunerChore exiting
10:07:02.914 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionChecker
regionserver/big-bdls02.audi.local/10.100.12.22:60020.compactionChecker exiting
10:07:02.880 AM INFO org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation
Closing zookeeper sessionid=0x253dfbf96e40006
10:07:02.942 AM INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher
MemStoreFlusher.1 exiting
10:07:02.955 AM INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher
MemStoreFlusher.0 exiting
10:07:02.914 AM INFO org.apache.hadoop.hbase.regionserver.ServerNonceManager$1
regionserver/big-bdls02.audi.local/10.100.12.22:60020.nonceCleaner exiting
10:07:02.995 AM INFO org.apache.zookeeper.ZooKeeper
Session: 0x253dfbf96e40006 closed
10:07:02.995 AM INFO org.apache.zookeeper.ClientCnxn
EventThread shut down
10:07:03.013 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer
stopping server big-bdls02.audi.local,60020,1459747837953; all regions closed.
10:07:03.451 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher
regionserver/big-bdls02.audi.local/10.100.12.22:60020.periodicFlusher exiting
10:07:03.514 AM INFO org.apache.hadoop.hbase.regionserver.Leases
regionserver/big-bdls02.audi.local/10.100.12.22:60020 closing leases
10:07:03.514 AM INFO org.apache.hadoop.hbase.regionserver.Leases
regionserver/big-bdls02.audi.local/10.100.12.22:60020 closed leases
10:07:03.678 AM INFO org.apache.hadoop.hbase.regionserver.LogRoller
LogRoller exiting.
10:07:03.684 AM INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread
Waiting for Split Thread to finish...
10:07:03.684 AM INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread
Waiting for Merge Thread to finish...
10:07:03.684 AM INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread
Waiting for Large Compaction Thread to finish...
10:07:03.685 AM INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread
Waiting for Small Compaction Thread to finish...
10:07:03.742 AM INFO org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation
Closing zookeeper sessionid=0x153dfbf94970005
10:07:03.762 AM INFO org.apache.zookeeper.ClientCnxn
EventThread shut down
10:07:03.762 AM INFO org.apache.zookeeper.ZooKeeper
Session: 0x153dfbf94970005 closed
10:07:03.763 AM INFO org.apache.hadoop.hbase.ipc.RpcServer
Stopping server on 60020
10:07:03.763 AM INFO org.apache.hadoop.hbase.ipc.RpcServer
RpcServer.listener,port=60020: stopping
10:07:03.962 AM INFO org.apache.hadoop.hbase.ipc.RpcServer
RpcServer.responder: stopped
10:07:03.982 AM INFO org.apache.hadoop.hbase.ipc.RpcServer
RpcServer.responder: stopping
10:07:04.305 AM INFO org.apache.hadoop.hbase.regionserver.Leases
regionserver/big-bdls02.audi.local/10.100.12.22:60020.leaseChecker closing leases
10:07:04.331 AM INFO org.apache.hadoop.hbase.regionserver.Leases
regionserver/big-bdls02.audi.local/10.100.12.22:60020.leaseChecker closed leases
10:07:04.365 AM INFO org.apache.zookeeper.ZooKeeper
Session: 0x253dfbf96e40004 closed
10:07:04.366 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer
stopping server big-bdls02.audi.local,60020,1459747837953; zookeeper connection closed.
10:07:04.366 AM INFO org.apache.hadoop.hbase.regionserver.HRegionServer
regionserver/big-bdls02.audi.local/10.100.12.22:60020 exiting
10:07:04.379 AM INFO org.apache.zookeeper.ClientCnxn
EventThread shut down
10:07:04.380 AM ERROR org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine
Region server exiting
java.lang.RuntimeException: HRegionServer Aborted
at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2537)
10:07:04.400 AM INFO org.apache.hadoop.hbase.regionserver.ShutdownHook
Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@1316bc42
10:07:04.494 AM INFO org.apache.hadoop.hbase.regionserver.ShutdownHook
Starting fs shutdown hook thread.
10:07:04.535 AM INFO org.apache.hadoop.hbase.regionserver.ShutdownHook
Shutdown hook finished.

 

 

Here is my log data,kindly give solution for this.

 

1 ACCEPTED SOLUTION

avatar
Mentor
When an RS exits, it logs a lot of content from the many threads it runs. Your log snippet is just the post-exit-decision messages, as you can note from the several "exiting" messages.

Please instead pass us your entire log file, or look for the entire line and its following exception stack trace matching the word "FATAL", which would be in a line well before these logs appear.

View solution in original post

1 REPLY 1

avatar
Mentor
When an RS exits, it logs a lot of content from the many threads it runs. Your log snippet is just the post-exit-decision messages, as you can note from the several "exiting" messages.

Please instead pass us your entire log file, or look for the entire line and its following exception stack trace matching the word "FATAL", which would be in a line well before these logs appear.