Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HBase client failing to connect to Kerberized HBase

avatar
Explorer

Here is the stacktrace:

Exception in thread "main" java.io.IOException: Login failure for hbase@EXAMPLE.COM from keytab /etc/security/keytabs/hbase.service.keytab at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1103) at org.apache.hadoop.security.UserGroupInformation$loginUserFromKeytabAndReturnUGI$0.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at hbase_test.run(hbase_test.groovy:23) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024) at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:923) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:906) at org.codehaus.groovy.runtime.InvokerHelper.runScript(InvokerHelper.java:410) at org.codehaus.groovy.runtime.InvokerHelper$runScript.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at hbase_test.main(hbase_test.groovy) Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687) at javax.security.auth.login.LoginContext.login(LoginContext.java:595) at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1092) ... 21 more

And this is the relevant part of the client:

Configuration conf = HBaseConfiguration.create(); conf.set("hadoop.security.authentication", "Kerberos"); UserGroupInformation.setConfiguration(conf) def userInfo = UserGroupInformation.loginUserFromKeytabAndReturnUGI("hbase@EXAMPLE.COM", args[0]);

UserGroupInformation.setLoginUser(userInfo)

1 ACCEPTED SOLUTION

avatar
Master Collaborator

Which user did you use to run the code ?

What's the output of the following command ?

klist -kt /etc/security/keytabs/hbase.service.keytab

Normally hbase.service.keytab should be used by user 'hbase'.

Please illustrate your use case in more detail.

Please take a look at

hbase-common/src/main/java/org/apache/hadoop/hbase/AuthUtil.java

View solution in original post

5 REPLIES 5

avatar
Master Collaborator

Which user did you use to run the code ?

What's the output of the following command ?

klist -kt /etc/security/keytabs/hbase.service.keytab

Normally hbase.service.keytab should be used by user 'hbase'.

Please illustrate your use case in more detail.

Please take a look at

hbase-common/src/main/java/org/apache/hadoop/hbase/AuthUtil.java

avatar
Explorer

Thanks for the klist suggestion, Ted. That and mqureshi's comment solved it for me.

avatar
Super Guru
@Mike Thomsen

hbas user is usually hbase/_HOST@REALM.COM. I don't see the host part of the principal. Is this how you have setup your hbase principal?

What are the permissions on your /etc/security/keytabs/hbase.service.keytab file?

avatar
Explorer

And you were correct! It's on an AWS cluster and part of the problem was that the principle had the internal interface associated with it so even when I added what I thought was the right host, didn't work until I followed Ted's advice to use klist.

avatar
Contributor

Hi,

I am using below principals for hbase kerberos authentication:

hbase.zookeeper.quorum=localhost
hbase.zookeeper.property.clientPort=2181
hadoop.security.authentication=kerberos
hbase.security.authentication=kerberos
hbase.master.kerberos.principal=zookeeper/localhost@EXAMPLE.COM
hbase.regionserver.kerberos.principal=zookeeper/localhost@EXAMPLE.COM
hbase.kerberos.principal=zookeeper/localhost@EXAMPLE.COM
hbase.kerberos.keytab=zkpr.keytab

Now, when i run my spark job on local it is not connecting to hbase, it shows error message:

Unable to connect to zookeeper/localhost@EXAMPLE.COM to zookeeper/localhost@EXAMPLE.COM.

I have done kinit zookeeper/localhost@EXAMPLE.COM -k -t zkpr.keytab and it is running fine.

Any help will be appreciated.