Support Questions

Find answers, ask questions, and share your expertise

Connecting to Kerberos-enabled hive via JDBC directly from Java

I have a Kerberos-enabled sandbox that I can connect to Hive via beeline after a kinit: !connect jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM

However, I need to connect directly from a Java program using the JDBC driver. A simple program fails to connect using the same URL that works for beeline above. I receive the following error:

Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM: GSS initiate failed
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at HiveJDBCTest.main(HiveJDBCTest.java:17)
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
... 5 more

Here is the simple program I am attempting to run:

import org.apache.hive.jdbc.HiveDriver;
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
public class HiveJDBCTest{
        public static void main(String[] args) throws SQLException{
                try {
                        Class.forName("org.apache.hive.jdbc.HiveDriver");
                } catch (ClassNotFoundException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                        System.exit(1);
                }
                Connection cnct = DriverManager.getConnection("jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM");
                Statement stmt = cnct.createStatement();
                String sql = "show tables;";
                System.out.println("Running: " + sql);
                ResultSet res = stmt.executeQuery(sql);
                if (res.next()) {
                        System.out.println(res.getString(1));
                }
        }
}

Any thoughts?

1 ACCEPTED SOLUTION

11 REPLIES 11

Master Collaborator

Did you do kinit on the shell where you run the Java client from?

Yup, I did. I have a ticket in the cache for the smoke user. This is the same user that I used to verify the connection with beeline. My understanding is that, like beeline, the JDBC driver should pick up my ticket from the cache without any intervention. Is that accurate?

Default principal: ambari-qa@EXAMPLE.COM
Valid starting     Expires            Service principal
10/21/15 15:48:20  10/22/15 15:48:20  krbtgt/EXAMPLE.COM@EXAMPLE.COM
renew until 10/21/15 15:48:20

Master Collaborator

For a single user case you may not have to do programmatic login, did you try including /etc/hive/conf in the classpath of your Java client?

No dice. Adding /etc/hive/conf/* to the classpath does not help. Is there a way programmatically tell Java to grab the existing credentials from the cache?

**EDIT** Corrected /etc/hive/* /etc/hive/conf/*

Master Collaborator

Couldn't add a longer reply for some reason so replied as an answer see below.

Contributor

its been a while, but this is at least a place to start looking at: when you grab a connection in a kerberos mode, you need to perform the following in the connection:

Subject current = Subject.getSubject(AccessController.getContext());

this.connection = (Connection) Subject.doAs(signedOnUserSubject, new PrivilegedExceptionAction<Object>() {

@Brandon Wilson

This may help ...Link

Thanks. Java (nor the JDBC driver) does not automatically pick up the cached ticket as beeline does. I added the following based on the link provided and authenticate within the program and it works.

org.apache.hadoop.conf.Configuration conf = new     org.apache.hadoop.conf.Configuration();
                    conf.set("hadoop.security.authentication", "Kerberos");
                    UserGroupInformation.setConfiguration(conf);
                    UserGroupInformation.loginUserFromKeytab("ambari-qa@EXAMPLE.COM", "/etc/security/keytabs/smokeuser.headless.keytab");

It can use cache if you use loginUserFromSubject.

Master Collaborator

Did you try adding /etc/hive/conf, it will be the location where your hive-site.xml is? AFAIK for a single user case you should not need to do anything with UGI object. For a multi-user case, you can do something like:

UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, keytabfile);
Connection con = (Connection) ugi.doAs(new PrivilegedExceptionAction<Object>() {
      public Object run() {
        Connection tcon = null;
        try {
          tcon = DriverManager.getConnection(connectionURL, user, passwd);
        } catch (SQLException e) {}
        return tcon;
      }
});

Refer to Hive Wiki.

New Contributor

I resolved the issue by using answer by @Hajime