question Re: Connecting to Kerberos-enabled hive via JDBC directly from Java in Archives of Support Questions (Read Only)

Connecting to Kerberos-enabled hive via JDBC directly from Java

bwilson — Wed, 21 Oct 2015 23:02:12 GMT

I have a Kerberos-enabled sandbox that I can connect to Hive via beeline after a kinit: !connect jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM

However, I need to connect directly from a Java program using the JDBC driver. A simple program fails to connect using the same URL that works for beeline above. I receive the following error:

Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM: GSS initiate failed
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at HiveJDBCTest.main(HiveJDBCTest.java:17)
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
... 5 more

Here is the simple program I am attempting to run:

import org.apache.hive.jdbc.HiveDriver;
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
public class HiveJDBCTest{
        public static void main(String[] args) throws SQLException{
                try {
                        Class.forName("org.apache.hive.jdbc.HiveDriver");
                } catch (ClassNotFoundException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                        System.exit(1);
                }
                Connection cnct = DriverManager.getConnection("jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM");
                Statement stmt = cnct.createStatement();
                String sql = "show tables;";
                System.out.println("Running: " + sql);
                ResultSet res = stmt.executeQuery(sql);
                if (res.next()) {
                        System.out.println(res.getString(1));
                }
        }
}

Any thoughts?

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

deepesh1 — Wed, 21 Oct 2015 23:06:31 GMT

Did you do kinit on the shell where you run the Java client from?

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

bwilson — Wed, 21 Oct 2015 23:10:34 GMT

Yup, I did. I have a ticket in the cache for the smoke user. This is the same user that I used to verify the connection with beeline. My understanding is that, like beeline, the JDBC driver should pick up my ticket from the cache without any intervention. Is that accurate?

Default principal: ambari-qa@EXAMPLE.COM
Valid starting     Expires            Service principal
10/21/15 15:48:20  10/22/15 15:48:20  krbtgt/EXAMPLE.COM@EXAMPLE.COM
renew until 10/21/15 15:48:20

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

hfaouaz — Wed, 21 Oct 2015 23:19:41 GMT

its been a while, but this is at least a place to start looking at: when you grab a connection in a kerberos mode, you need to perform the following in the connection:

Subject current = Subject.getSubject(AccessController.getContext());

this.connection = (Connection) Subject.doAs(signedOnUserSubject, new PrivilegedExceptionAction<Object>() {

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

nsabharwal — Wed, 21 Oct 2015 23:26:03 GMT

@Brandon Wilson

This may help ...Link

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

bwilson — Wed, 21 Oct 2015 23:37:43 GMT

Thanks. Java (nor the JDBC driver) does not automatically pick up the cached ticket as beeline does. I added the following based on the link provided and authenticate within the program and it works.

org.apache.hadoop.conf.Configuration conf = new     org.apache.hadoop.conf.Configuration();
                    conf.set("hadoop.security.authentication", "Kerberos");
                    UserGroupInformation.setConfiguration(conf);
                    UserGroupInformation.loginUserFromKeytab("ambari-qa@EXAMPLE.COM", "/etc/security/keytabs/smokeuser.headless.keytab");

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

deepesh1 — Wed, 21 Oct 2015 23:40:56 GMT

For a single user case you may not have to do programmatic login, did you try including /etc/hive/conf in the classpath of your Java client?

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

bwilson — Thu, 22 Oct 2015 00:42:50 GMT

No dice. Adding /etc/hive/conf/* to the classpath does not help. Is there a way programmatically tell Java to grab the existing credentials from the cache?

**EDIT** Corrected /etc/hive/* /etc/hive/conf/*

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

deepesh1 — Thu, 22 Oct 2015 01:17:06 GMT

Did you try adding /etc/hive/conf, it will be the location where your hive-site.xml is? AFAIK for a single user case you should not need to do anything with UGI object. For a multi-user case, you can do something like:

UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, keytabfile);
Connection con = (Connection) ugi.doAs(new PrivilegedExceptionAction<Object>() {
      public Object run() {
        Connection tcon = null;
        try {
          tcon = DriverManager.getConnection(connectionURL, user, passwd);
        } catch (SQLException e) {}
        return tcon;
      }
});

Refer to Hive Wiki.

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

deepesh1 — Thu, 22 Oct 2015 01:17:37 GMT

Couldn't add a longer reply for some reason so replied as an answer see below.

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

hosako — Tue, 22 Mar 2016 21:00:03 GMT

It can use cache if you use loginUserFromSubject.

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

madhura_verma — Mon, 28 Nov 2016 21:47:33 GMT

I resolved the issue by using answer by @Hajime