Reply
SL
Explorer
Posts: 9
Registered: ‎06-08-2015

HBase Thrift Server crash with python

[ Edited ]

When using python with happybase   ,the code like below  ,it always causes the thrift server crash ?Pleae help?

 

import happybase
import time
import os
import re
import sys
import datetime
print ('Connect XXX.XXX.XXX.XXX:9090')
connection = happybase.Connection(host='XXX.XXX.XXX.XXX',port=9090 )
print ('open table wiptable')
table = connection.table('wiptable')
for key, data in table.scan(row_start='KG'):
print key, data
connection.close()
##in python client log##
Traceback (most recent call last):
File "D:\Python27\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 323, in RunScript
debugger.run(codeObject, __main__.__dict__, start_stepping=0)
File "D:\Python27\Lib\site-packages\pythonwin\pywin\debugger\__init__.py", line 60, in run
_GetCurrentDebugger().run(cmd, globals,locals, start_stepping)
File "D:\Python27\Lib\site-packages\pythonwin\pywin\debugger\debugger.py", line 654, in run
exec cmd in globals, locals
File "D:\Python27\steven01.py", line 1, in <module>
import happybase
File "D:\Python27\lib\site-packages\happybase\table.py", line 409, in scan
self.connection.client.scannerClose(scan_id)
File "D:\Python27\lib\site-packages\happybase\hbase\Hbase.py", line 2290, in scannerClose
self.recv_scannerClose()
File "D:\Python27\lib\site-packages\happybase\hbase\Hbase.py", line 2301, in recv_scannerClose
(fname, mtype, rseqid) = self._iprot.readMessageBegin()
File "D:\Python27\lib\site-packages\thrift\protocol\TBinaryProtocol.py", line 126, in readMessageBegin
sz = self.readI32()
File "D:\Python27\lib\site-packages\thrift\protocol\TBinaryProtocol.py", line 206, in readI32
buff = self.trans.readAll(4)
File "D:\Python27\lib\site-packages\thrift\transport\TTransport.py", line 58, in readAll
chunk = self.read(sz - have)
File "D:\Python27\lib\site-packages\thrift\transport\TTransport.py", line 159, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
File "D:\Python27\lib\site-packages\thrift\transport\TSocket.py", line 120, in read
message='TSocket read 0 bytes')
TTransportException: TSocket read 0 bytes

 

 

#in hbase thrift server  log#

11:41:32.782 AM	INFO	org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper	
Process identifier=hconnection-0x50362ea3 connecting to ZooKeeper ensemble=hdp03:2181,hdp01:2181,hdp02:2181
11:41:32.783 AM	INFO	org.apache.zookeeper.ZooKeeper	
Initiating client connection, connectString=hdp03:2181,hdp01:2181,hdp02:2181 sessionTimeout=180000 watcher=hconnection-0x50362ea30x0, quorum=hdp03:2181,hdp01:2181,hdp02:2181, baseZNode=/hbase
11:41:32.785 AM	INFO	org.apache.zookeeper.ClientCnxn	
Opening socket connection to server hdp02/10.80.9.190:2181. Will not attempt to authenticate using SASL (unknown error)
11:41:32.786 AM	INFO	org.apache.zookeeper.ClientCnxn	
Socket connection established, initiating session, client: /10.80.9.190:48129, server: hdp02/10.80.9.190:2181
11:41:32.800 AM	INFO	org.apache.zookeeper.ClientCnxn	
Session establishment complete on server hdp02/10.80.9.190:2181, sessionid = 0x350c5a571770fb5, negotiated timeout = 180000
11:41:32.802 AM	DEBUG	org.apache.hadoop.hbase.ipc.AbstractRpcClient	
Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@654485dc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null
11:41:32.817 AM	DEBUG	org.apache.hadoop.hbase.ipc.AbstractRpcClient	
Use SIMPLE authentication for service ClientService, sasl=false
11:41:32.818 AM	DEBUG	org.apache.hadoop.hbase.ipc.AbstractRpcClient	
Connecting to hdp03/10.80.9.191:60020
11:41:32.846 AM	DEBUG	org.apache.hadoop.hbase.ipc.AbstractRpcClient	
Use SIMPLE authentication for service ClientService, sasl=false
11:41:32.847 AM	DEBUG	org.apache.hadoop.hbase.ipc.AbstractRpcClient	
Connecting to hdp02/10.80.9.190:60020
11:41:33.209 AM	DEBUG	org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler	
scannerGetList: id=0
11:41:33.999 AM	DEBUG	org.apache.hadoop.fs.azure.NativeAzureFileSystem	
finalize() called.
11:41:34 AM	DEBUG	org.apache.hadoop.fs.azure.NativeAzureFileSystem	
finalize() called.
Highlighted
Posts: 1,896
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: HBase Thrift Server crash with python

Can you check what mode of Thrift server your HBase Thrift Server is running with? If it is set to use framed transport, apply the same on your happybase client: http://happybase.readthedocs.org/en/latest/api.html#happybase.Connection

If you are using HTTP mode for use with Hue's HBase app, then happybase would not work (You can run another duplicated HBase Thrift Server on another host in regular binary thrift mode (framed or buffered), to cater to happybase clients specifically).
Announcements

Our community is getting a little larger. And a lot better.


Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.