Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Connecting to Kerberos-enabled hive via JDBC directly from Java

avatar

I have a Kerberos-enabled sandbox that I can connect to Hive via beeline after a kinit: !connect jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM

However, I need to connect directly from a Java program using the JDBC driver. A simple program fails to connect using the same URL that works for beeline above. I receive the following error:

Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM: GSS initiate failed
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at HiveJDBCTest.main(HiveJDBCTest.java:17)
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
... 5 more

Here is the simple program I am attempting to run:

import org.apache.hive.jdbc.HiveDriver;
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
public class HiveJDBCTest{
        public static void main(String[] args) throws SQLException{
                try {
                        Class.forName("org.apache.hive.jdbc.HiveDriver");
                } catch (ClassNotFoundException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                        System.exit(1);
                }
                Connection cnct = DriverManager.getConnection("jdbc:hive2://localhost:10000/default;principal=hive/sandbox.hortonworks.com@EXAMPLE.COM");
                Statement stmt = cnct.createStatement();
                String sql = "show tables;";
                System.out.println("Running: " + sql);
                ResultSet res = stmt.executeQuery(sql);
                if (res.next()) {
                        System.out.println(res.getString(1));
                }
        }
}

Any thoughts?

1 ACCEPTED SOLUTION

avatar
Master Mentor
11 REPLIES 11

avatar

Did you do kinit on the shell where you run the Java client from?

avatar

Yup, I did. I have a ticket in the cache for the smoke user. This is the same user that I used to verify the connection with beeline. My understanding is that, like beeline, the JDBC driver should pick up my ticket from the cache without any intervention. Is that accurate?

Default principal: ambari-qa@EXAMPLE.COM
Valid starting     Expires            Service principal
10/21/15 15:48:20  10/22/15 15:48:20  krbtgt/EXAMPLE.COM@EXAMPLE.COM
renew until 10/21/15 15:48:20

avatar

For a single user case you may not have to do programmatic login, did you try including /etc/hive/conf in the classpath of your Java client?

avatar

No dice. Adding /etc/hive/conf/* to the classpath does not help. Is there a way programmatically tell Java to grab the existing credentials from the cache?

**EDIT** Corrected /etc/hive/* /etc/hive/conf/*

avatar

Couldn't add a longer reply for some reason so replied as an answer see below.

avatar
Rising Star

its been a while, but this is at least a place to start looking at: when you grab a connection in a kerberos mode, you need to perform the following in the connection:

Subject current = Subject.getSubject(AccessController.getContext());

this.connection = (Connection) Subject.doAs(signedOnUserSubject, new PrivilegedExceptionAction<Object>() {

avatar
Master Mentor
@Brandon Wilson

This may help ...Link

avatar

Thanks. Java (nor the JDBC driver) does not automatically pick up the cached ticket as beeline does. I added the following based on the link provided and authenticate within the program and it works.

org.apache.hadoop.conf.Configuration conf = new     org.apache.hadoop.conf.Configuration();
                    conf.set("hadoop.security.authentication", "Kerberos");
                    UserGroupInformation.setConfiguration(conf);
                    UserGroupInformation.loginUserFromKeytab("ambari-qa@EXAMPLE.COM", "/etc/security/keytabs/smokeuser.headless.keytab");

avatar

It can use cache if you use loginUserFromSubject.