Connecting to Kerberos-enabled hive via JDBC directly from Java

[#1]

I have a Kerberos-enabled sandbox that I can connect to Hive via beeline after a kinit: !connect jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM

However, I need to connect directly from a Java program using the JDBC driver. A simple program fails to connect using the same URL that works for beeline above. I receive the following error:

Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM: GSS initiate failed
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at HiveJDBCTest.main(HiveJDBCTest.java:17)
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
... 5 more

Here is the simple program I am attempting to run:

import org.apache.hive.jdbc.HiveDriver;
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
public class HiveJDBCTest{
        public static void main(String[] args) throws SQLException{
                try {
                        Class.forName("org.apache.hive.jdbc.HiveDriver");
                } catch (ClassNotFoundException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                        System.exit(1);
                }
                Connection cnct = DriverManager.getConnection("jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM");
                Statement stmt = cnct.createStatement();
                String sql = "show tables;";
                System.out.println("Running: " + sql);
                ResultSet res = stmt.executeQuery(sql);
                if (res.next()) {
                        System.out.println(res.getString(1));
                }
        }
}

Any thoughts?

[#2]

Did you do kinit on the shell where you run the Java client from?

[#3]

Yup, I did. I have a ticket in the cache for the smoke user. This is the same user that I used to verify the connection with beeline. My understanding is that, like beeline, the JDBC driver should pick up my ticket from the cache without any intervention. Is that accurate?

Default principal: ambari-qa@EXAMPLE.COM
Valid starting     Expires            Service principal
10/21/15 15:48:20  10/22/15 15:48:20  krbtgt/EXAMPLE.COM@EXAMPLE.COM
renew until 10/21/15 15:48:20

[#4]

For a single user case you may not have to do programmatic login, did you try including /etc/hive/conf in the classpath of your Java client?

[#5]

No dice. Adding /etc/hive/conf/* to the classpath does not help. Is there a way programmatically tell Java to grab the existing credentials from the cache?

**EDIT** Corrected /etc/hive/* /etc/hive/conf/*

[#6]

Couldn't add a longer reply for some reason so replied as an answer see below.

[#7]

its been a while, but this is at least a place to start looking at: when you grab a connection in a kerberos mode, you need to perform the following in the connection:

Subject current = Subject.getSubject(AccessController.getContext());

this.connection = (Connection) Subject.doAs(signedOnUserSubject, new PrivilegedExceptionAction<Object>() {

[#8 · Accepted solution]

@Brandon Wilson

This may help ...Link

[#9]

Thanks. Java (nor the JDBC driver) does not automatically pick up the cached ticket as beeline does. I added the following based on the link provided and authenticate within the program and it works.

org.apache.hadoop.conf.Configuration conf = new     org.apache.hadoop.conf.Configuration();
                    conf.set("hadoop.security.authentication", "Kerberos");
                    UserGroupInformation.setConfiguration(conf);
                    UserGroupInformation.loginUserFromKeytab("ambari-qa@EXAMPLE.COM", "/etc/security/keytabs/smokeuser.headless.keytab");

[#10]

It can use cache if you use loginUserFromSubject.