Start and test HBASE thrift server in a kerberised environment

[#1]

Assuming you start with a kerberized HDP cluster with Hbase installed.

First check what your service principal is i.e.

klist -kt /etc/security/keytabs/hbase.service.keytab
Keytab name: FILE:hbase.service.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 12/20/16 13:51:21 hbase/hdp252.hdp@HWX.COM
   2 12/20/16 13:51:21 hbase/hdp252.hdp@HWX.COM
   2 12/20/16 13:51:21 hbase/hdp252.hdp@HWX.COM
   2 12/20/16 13:51:21 hbase/hdp252.hdp@HWX.COM
   2 12/20/16 13:51:21 hbase/hdp252.hdp@HWX.COM

In Ambari head to Hbase -> Configs -> Advanced -> Custom Hbase-Site.xml and add the following new parameters with the keytab / principal substituted:

hbase.thrift.security.qop=auth
hbase.thrift.support.proxyuser=true
hbase.regionserver.thrift.http=true
hbase.thrift.keytab.file=/etc/security/keytabs/hbase.service.keytab 
hbase.thrift.kerberos.principal=hbase/_HOST@HWX.COM 
hbase.security.authentication.spnego.kerberos.keytab=/etc/security/keytabs/spnego.service.keytab 
hbase.security.authentication.spnego.kerberos.principal=HTTP/_HOST@HDP.COM 

Check that the following are set in HDFS and if not, add them to 'Custom core-site.xml'

hadoop.proxyuser.hbase.groups=*
hadoop.proxyuser.hbase.hosts=*

Restart the affected HBase & HDFS services.

On the command line on the HBase master, kinit with the service keytab and start the thrift server:

su - hbase
kinit -kt hbase.service.keytab hbase/hdp252.hdp@HWX.COM/usr/hdp/current/hbase-master/bin/hbase-daemon.sh start thrift --infoport 8086

The parameter we set earlier 'hbase.regionserver.thrift.http=true' indicates that the thrift server will be started in http mode. To start in binary mode set this to false.

Logs are written to /var/log/hbase and you should see a running process

To test the thrift server in http mode the syntax is:

hbase org.apache.hadoop.hbase.thrift.HttpDoAsClient hdp252 9090 hbase true

to test in binary mode the syntax is:

hbase org.apache.hadoop.hbase.thrift.DemoClient hdp252 9090 true

[#2]

Hi -

We have a kerberized cluster HDP 2.5.3 and I have followed your instructions to the T and while I have no problems with Hive, Job Browser, & File Browser in HUE, I continue to get this error when trying to access HBASE tables in HUE:

Api Error: Could not start SASL: Error in sasl_client_start (-1) SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server not found in Kerberos database)

[#3]

I forgot to state that I have the hue user set up to impersonate/proxy in the core-site file as well.

[#4]

Shouldn't bellow use Spnego one?

1. hbase.thrift.keytab.file=/etc/security/keytabs/hbase.service.keytab
2. hbase.thrift.kerberos.principal=hbase/_HOST@HWX.COM

Otherwise, I couldn't make "hbase org.apache.hadoop.hbase.thrift.HttpDoAsClient" work from another node.

Or am I missing something else?

[#5]

Thanks for this article. Everything works fine, except that my thrift server fails to behave properly after hbase user kerberos ticket expiration (10h in my case). Is there a way to automatically refresh/renew ticket so that my thrift server runs endlessly ?

Thanks