Support Questions
Find answers, ask questions, and share your expertise

Issue when trying to reach HBase thrift server from my edge server

Rising Star


I use a kerberized 2.6.3 HDP cluster comprising 3 HBase master nodes (named p21, p22 & p23).

On one of these master nodes (p23) , I started up a thrift server following explanations from :

Everything went fine and when I launch (being "hbase" user) the DemoClient from one of the 3 master nodes ("hbase org.apache.hadoop.hbase.thrift.DemoClient p23 9090 true"), it executes successfully (even when launching from the 2 master nodes that don't host the thrift server)

Anyway, when I try to execute some client code from an edge node (and from another "non hbase" user) to access my thrift server, it fails to executes properly :

I can see many instances of following error in thrift server logs :

2018-01-25 14:12:40,051 INFO [thrift-worker-0] client.RpcRetryingCaller: Call exception, tries=10, retries=35, started=48576 ms ago, cancelled=false, org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to p21/10.XXX.XXX.XXX:16000 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to p21/10.XXX.XXX.XXX:16000 is closing. Call id=10, waitTime=7

...And after a while, my client fails and return same kind of error...

It's worth saying that I can successfully execute "hbase shell" queries from this edge node, so I guess my "hbase-site.xml" file is correct...

Any idea why this error can happen only for "thrift-based" queries ?

Thanks for your help



Rising Star

I finally figured out what happened : In fact, if hbase user has a valid kerberos ticket when he starts the thrift server, eveything works fine during the ticket lifetime (10h in my case), but thrift server fails to behave properly after this ticket expired...Is there a way to configure something on thrift server side to automatically refresh/renew the kerberos ticket ?

At the moment, I must stop/restart the thrift server every 10h...

Thanks for your advices