beeline returns "Failed to find any Kerberos tgt" after enabling kerberos

[#1]

Basically followed the instruction in http://hortonworks.com/blog/enabling-kerberos-hdp-active-directory-integration/

From ambari, everything looks OK but beeline command fails with "GSS initiate failed (state=08S01,code=0)"

Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
        at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
        ... 28 more

Checked time is synchronized and resolving IP/hostname is correct.

Built another HDP 2.3.2 and Ambari 2.1.2 without AD this time (simple MIT KDC)

And Ambari looks OK but, again, beeline fails with same error.

How I'm starting beeline is like below:

su - hive

beeline -u "jdbc:hive2://hiveserver2_fqdn:10000/default;principal=hive/hiveserver2_fqdn@MY_REALM"

I think i'm forgetting some setting...

I appreciate any advice from you.

Thank you

[#2 · Accepted solution]

@Hajime

Failed to find Kerberos tgt

No valid credentials provided

Make sure that you have ticket

klist

kint -kt ( please check official docs on Kerberos setup)

[#3]

That's something i'm not sure. If i use hive/FQDN@MY_REALM with kinit, i can use "hdfs dfs -ls" command, then I started beeline, then get this error.

[#4]

@Hajime

Paste the output of klist

And search for Kerberos in hive configs by using filter option and paste the screenshot or config settings here

[#5]

@Hajime Based on the error messages , I can see that Kerberos is not setup correctly.

Please follow this

[#6]

Your beeline command is fine and should work.

Could you please check your Namenode and Hive log to see if there are any kerberos-related issues? I have seen clusters with a green status in Ambari, but the log files were full of Kerberos authentication failures.

[#7]

I have the same probem.

Lines from hive-site.xml:

<property> <name>hive.server2.authentication</name> <value>KERBEROS</value> </property> <property> <name>hive.server2.authentication.kerberos.keytab</name> <value>/etc/security/keytabs/hive.service.keytab</value> </property> <property> <name>hive.server2.authentication.kerberos.principal</name> <value>hive/_HOST@EXAMPLE.COM</value> </property>

[margusja@sandbox ~]$ kinit -R

[margusja@sandbox ~]$ klist -f Ticket cache: FILE:/tmp/krb5cc_1024 Default principal: margusja@EXAMPLE.COM Valid starting Expires Service principal 01/10/16 08:13:21 01/11/16 08:13:21 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 01/17/16 08:07:43, Flags: FRIT

I can use hdfs:

[margusja@sandbox ~]$ hdfs dfs -ls / Found 9 items drwxrwxrwx - yarn hadoop 0 2016-01-10 07:19 /app-logs drwxr-xr-x - hdfs hdfs 0 2015-10-27 13:19 /apps drwxr-xr-x - hdfs hdfs 0 2015-10-27 13:06 /demo drwxr-xr-x - hdfs hdfs 0 2015-10-27 12:39 /hdp drwxr-xr-x - mapred hdfs 0 2015-10-27 12:39 /mapred drwxrwxrwx - mapred hadoop 0 2015-10-27 12:40 /mr-history drwxrwxrwx - hdfs hdfs 0 2015-10-27 13:12 /ranger drwxrwxrwx - hdfs hdfs 0 2015-10-27 12:54 /tmp drwxr-xr-x - hdfs hdfs 0 2016-01-09 19:39 /user

But I can not use beeline:

http://pastebin.com/7AyhpYsq

Br, Margusja

[#8]

Same issue as @Margus Roo, I use sandbox 2.4

[#9]

I resolved this problem after debugging kerberos traffic. I set export HADOOP_OPTS="-Dsun.security.krb5.debug=true" in command line and then used beeline command. It showed what was wrong with TGT.

[#10]

How do you fix the problem?

Can you please post the detail steps?

Thanks,