Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Connecting to Kerberos-enabled hive via JDBC directly from Java

SOLVED Go to solution
Highlighted

Connecting to Kerberos-enabled hive via JDBC directly from Java

I have a Kerberos-enabled sandbox that I can connect to Hive via beeline after a kinit: !connect jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM

However, I need to connect directly from a Java program using the JDBC driver. A simple program fails to connect using the same URL that works for beeline above. I receive the following error:

Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM: GSS initiate failed
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at HiveJDBCTest.main(HiveJDBCTest.java:17)
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
... 5 more

Here is the simple program I am attempting to run:

import org.apache.hive.jdbc.HiveDriver;
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
public class HiveJDBCTest{
        public static void main(String[] args) throws SQLException{
                try {
                        Class.forName("org.apache.hive.jdbc.HiveDriver");
                } catch (ClassNotFoundException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                        System.exit(1);
                }
                Connection cnct = DriverManager.getConnection("jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM");
                Statement stmt = cnct.createStatement();
                String sql = "show tables;";
                System.out.println("Running: " + sql);
                ResultSet res = stmt.executeQuery(sql);
                if (res.next()) {
                        System.out.println(res.getString(1));
                }
        }
}

Any thoughts?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

@Brandon Wilson

This may help ...Link

11 REPLIES 11

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

Master Collaborator

Did you do kinit on the shell where you run the Java client from?

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

Yup, I did. I have a ticket in the cache for the smoke user. This is the same user that I used to verify the connection with beeline. My understanding is that, like beeline, the JDBC driver should pick up my ticket from the cache without any intervention. Is that accurate?

Default principal: ambari-qa@EXAMPLE.COM
Valid starting     Expires            Service principal
10/21/15 15:48:20  10/22/15 15:48:20  krbtgt/EXAMPLE.COM@EXAMPLE.COM
renew until 10/21/15 15:48:20

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

Master Collaborator

For a single user case you may not have to do programmatic login, did you try including /etc/hive/conf in the classpath of your Java client?

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

No dice. Adding /etc/hive/conf/* to the classpath does not help. Is there a way programmatically tell Java to grab the existing credentials from the cache?

**EDIT** Corrected /etc/hive/* /etc/hive/conf/*

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

Master Collaborator

Couldn't add a longer reply for some reason so replied as an answer see below.

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

Contributor

its been a while, but this is at least a place to start looking at: when you grab a connection in a kerberos mode, you need to perform the following in the connection:

Subject current = Subject.getSubject(AccessController.getContext());

this.connection = (Connection) Subject.doAs(signedOnUserSubject, new PrivilegedExceptionAction<Object>() {

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

@Brandon Wilson

This may help ...Link

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

Thanks. Java (nor the JDBC driver) does not automatically pick up the cached ticket as beeline does. I added the following based on the link provided and authenticate within the program and it works.

org.apache.hadoop.conf.Configuration conf = new     org.apache.hadoop.conf.Configuration();
                    conf.set("hadoop.security.authentication", "Kerberos");
                    UserGroupInformation.setConfiguration(conf);
                    UserGroupInformation.loginUserFromKeytab("ambari-qa@EXAMPLE.COM", "/etc/security/keytabs/smokeuser.headless.keytab");

Re: Connecting to Kerberos-enabled hive via JDBC directly from Java

It can use cache if you use loginUserFromSubject.