- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Hive JDBC driver with keytab authentication
- Labels:
-
Apache Hive
Created 03-14-2018 09:21 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, I am using a 3rd party ETL application that I would like to connect to Hive as a datasource, using the Hive JDBC driver. Our Hive is kerberized so it needs a kerberos ticket to be able to connect. I am wondering if there is any way to automatically trigger a login from keytab when the application tries to connect to Hive. The tool itself is Java-based so I can set Java system properties when it starts, and I can also add config settings to the Hive JDBC connection string.
I've been able to get this to work by running a kinit before the process and setting -Djavax.security.auth.useSubjectCredsOnly=false, but I'd like to avoid this because it seems to be unreliable (sometimes when connecting to the DB the Java kerberos system ends up prompting for a username and password, even though the Kinit just ran, and I don't know why).
I have also tried setting a custom JAAS conf file as:
com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useKeyTab=true
keyTab=(path to keytab)
principal=(prinicpal)
debug=true; };
This doesn't seem to do anything. My guess is that Hadoop UserGroupInformation class, which the JDBC driver uses to handle security, ignores these JAAS settings.
So to sum it up - is there any way I can get the JDBC driver to automatically login from keytab, by either setting Java system properties or changing the JDBC connection string?
Created 03-16-2018 04:37 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Check this out. https://github.com/rajkrrsingh/HiveServer2JDBCSample/blob/master/src/main/java/HiveJDBCOverHTTP.java
Hope it helps.
Created 03-17-2018 06:45 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you have Knox you can use it to encapsulate Kerberos authentication and use username/password.
Thanks
Created 03-19-2018 08:18 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks @dvillarreal and @Abdelkrim Hadjidj, it turned out that using a gss-jaas.conf file does work to auto-login from keytab, I just had to make sure to also set
-Djavax.security.auth.useSubjectCredsOnly=false
in the startup options of the ETL tool.
I forgot to mention that this was in a Windows environment, so this could help with using the Hive JDBC driver with other Windows tools.
Created 12-18-2018 03:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you already have a TGT available in cache, then below code worked for me
import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.SQLException; import java.sql.Statement; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.security.UserGroupInformation; class HiveBeelineTest { public static void main(String args[]) throws ClassNotFoundException, SQLException { Class.forName("org.apache.hive.jdbc.HiveDriver"); Configuration conf = new Configuration(); conf.set("hadoop.security.authentication", "Kerberos"); UserGroupInformation.setConfiguration(conf); Connection con = DriverManager.getConnection( "jdbc:hive2://mynode.foobar.com:10000/default;principal=hive/mynode.foobar.com@FOOBAR.COM") Statement sqlstatement = con.createStatement(); ResultSet result = sqlstatement.executeQuery("select test_column from test_table"); while (result.next()) { System.out.println(result.getString(1)); } } }