Reply
Explorer
Posts: 11
Registered: ‎04-13-2015

HBase Thrift Server Down

Hello World,

 

I see HBase Thrift Server goes in CDH5.3.2 with below error in logs, can somebody help me to understand issue.

 

ERROR : 

 

Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset
	at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
	at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:283)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
	at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
	at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
	at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
	at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
	... 4 more
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:189)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
	... 10 more

 

Thanks,

Venu

Highlighted
Posts: 1,903
Kudos: 435
Solutions: 305
Registered: ‎07-31-2013

Re: HBase Thrift Server Down

Its not possible to diagnose an entire issue with just that trace, but a connection reset likely means the client disconnected before the server could respond (possibly as a result of a timeout from the client's perspective, and a late result from the server then would print this when trying to write it back).
Posts: 1,903
Kudos: 435
Solutions: 305
Registered: ‎07-31-2013

Re: HBase Thrift Server Down

But this error will not cause a role crash as your title describes. It likely went down either cause framed transport is not enabled and a bad request reached the thrift server, or maybe an OOME (see stdout of the crashed process).