Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HBase client failing to connect to Kerberized HBase

Solved Go to solution

HBase client failing to connect to Kerberized HBase

New Contributor

Here is the stacktrace:

Exception in thread "main" java.io.IOException: Login failure for hbase@EXAMPLE.COM from keytab /etc/security/keytabs/hbase.service.keytab at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1103) at org.apache.hadoop.security.UserGroupInformation$loginUserFromKeytabAndReturnUGI$0.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at hbase_test.run(hbase_test.groovy:23) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024) at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:923) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:906) at org.codehaus.groovy.runtime.InvokerHelper.runScript(InvokerHelper.java:410) at org.codehaus.groovy.runtime.InvokerHelper$runScript.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at hbase_test.main(hbase_test.groovy) Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687) at javax.security.auth.login.LoginContext.login(LoginContext.java:595) at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1092) ... 21 more

And this is the relevant part of the client:

Configuration conf = HBaseConfiguration.create(); conf.set("hadoop.security.authentication", "Kerberos"); UserGroupInformation.setConfiguration(conf) def userInfo = UserGroupInformation.loginUserFromKeytabAndReturnUGI("hbase@EXAMPLE.COM", args[0]);

UserGroupInformation.setLoginUser(userInfo)

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: HBase client failing to connect to Kerberized HBase

Super Collaborator

Which user did you use to run the code ?

What's the output of the following command ?

klist -kt /etc/security/keytabs/hbase.service.keytab

Normally hbase.service.keytab should be used by user 'hbase'.

Please illustrate your use case in more detail.

Please take a look at

hbase-common/src/main/java/org/apache/hadoop/hbase/AuthUtil.java

5 REPLIES 5
Highlighted

Re: HBase client failing to connect to Kerberized HBase

Super Collaborator

Which user did you use to run the code ?

What's the output of the following command ?

klist -kt /etc/security/keytabs/hbase.service.keytab

Normally hbase.service.keytab should be used by user 'hbase'.

Please illustrate your use case in more detail.

Please take a look at

hbase-common/src/main/java/org/apache/hadoop/hbase/AuthUtil.java

Re: HBase client failing to connect to Kerberized HBase

New Contributor

Thanks for the klist suggestion, Ted. That and mqureshi's comment solved it for me.

Re: HBase client failing to connect to Kerberized HBase

Super Guru
@Mike Thomsen

hbas user is usually hbase/_HOST@REALM.COM. I don't see the host part of the principal. Is this how you have setup your hbase principal?

What are the permissions on your /etc/security/keytabs/hbase.service.keytab file?

Re: HBase client failing to connect to Kerberized HBase

New Contributor

And you were correct! It's on an AWS cluster and part of the problem was that the principle had the internal interface associated with it so even when I added what I thought was the right host, didn't work until I followed Ted's advice to use klist.

Re: HBase client failing to connect to Kerberized HBase

New Contributor

Hi,

I am using below principals for hbase kerberos authentication:

hbase.zookeeper.quorum=localhost
hbase.zookeeper.property.clientPort=2181
hadoop.security.authentication=kerberos
hbase.security.authentication=kerberos
hbase.master.kerberos.principal=zookeeper/localhost@EXAMPLE.COM
hbase.regionserver.kerberos.principal=zookeeper/localhost@EXAMPLE.COM
hbase.kerberos.principal=zookeeper/localhost@EXAMPLE.COM
hbase.kerberos.keytab=zkpr.keytab

Now, when i run my spark job on local it is not connecting to hbase, it shows error message:

Unable to connect to zookeeper/localhost@EXAMPLE.COM to zookeeper/localhost@EXAMPLE.COM.

I have done kinit zookeeper/localhost@EXAMPLE.COM -k -t zkpr.keytab and it is running fine.

Any help will be appreciated.