Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Executing Hive queries through oozie java action on kerberized cluster

Highlighted

Executing Hive queries through oozie java action on kerberized cluster

New Contributor

Hi,

I am trying to execute hive queries through oozie Java action on kerberized cluster, however I am getting below exception:

>>> Invoking Main class now >>>
Launch time = 1458542514082
Job launch time = 1458542514082 mapreduce.job.tags = oozie-2e8d7ed9fc7551a353667830e09bef2b
Main class        : com.citiustech.main.Test
Arguments         :

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw exception, org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://10.60.2.26:10000/default;principal=hive/example-qa1-dn2@EXAMPLE.com;auth=NOSASL: GSS initiate failed
org.apache.oozie.action.hadoop.JavaMainException: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://10.60.2.26:10000/default;principal=hive/example-qa1-dn2@EXAMPLE.com;auth=NOSASL: GSS initiate failed
	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
	at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://10.60.2.26:10000/default;principal=hive/example-qa1-dn2@EXAMPLE.com;auth=NOSASL: GSS initiate failed
	at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:391)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:471)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:481)
	at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:491)
	at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:497)
	at com.citiustech.main.Test.main(Test.java:26)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:55)
	... 15 more
Caused by: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://10.60.2.26:10000/default;principal=hive/example-qa1-dn2@EXAMPLE.com;auth=NOSASL: GSS initiate failed
	at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:210)
	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:156)
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
	at java.sql.DriverManager.getConnection(DriverManager.java:571)
	at java.sql.DriverManager.getConnection(DriverManager.java:187)
	at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:153)
	at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:144)
	at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:155)
	at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:120)
	at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
	at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
	... 26 more
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
	at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
	at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
	at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
	at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
	at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
	at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
	at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:185)
	... 36 more

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher, uploading action data to HDFS sequence file: hdfs://example-qa1-nn:8020/user/admin/oozie-oozi/0000000-160317141545276-oozie-oozi-W/javaAction--java/action-data.seq

Oozie Launcher ends

Please find below workflow xml specifying oozie java action and java program for executing hive queries:

Workflow.xml:

<workflow-app name="WorkFlowForJavaActionToExecuteHiveQuery" xmlns="uri:oozie:workflow:0.2.5">
		<credentials>
			<credential name='hive_credentials' type='hcat'>
				<property>
					<name>hcat.metastore.uri</name>
					<value>thrift://example-qa1-dn2:9083</value>
				</property>
				<property>
					<name>hcat.metastore.principal</name>
					<value>hive/_HOST@EXAMPLE.COM</value>
				</property>
			</credential>
		</credentials>
		<start to="javaAction"/>
		<action name="javaAction" cred="hive_credentials">
			<java>
				<job-tracker>${jobTracker}</job-tracker>
				<name-node>${nameNode}</name-node>
				<main-class>com.demo.main.Test</main-class>
			</java>
			<ok to="end"/>
			<error to="fail"/>
		</action>
		<kill name="fail">
			<message>Job failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
		</kill>
		<end name="end"/>
	</workflow-app>

Java Program:

package com.demo.main;
import java.io.IOException;
import org.apache.hadoop.security.UserGroupInformation;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
public class Test {
	public static void main(String[] args) throws IOException {
		UserGroupInformation.loginUserFromKeytab("admin@EXAMPLE.COM", "/etc/security/keytabs/admin.keytab");
		org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
		conf.set("hadoop.security.authentication", "Kerberos");
		UserGroupInformation.setConfiguration(conf);
		UserGroupInformation.loginUserFromKeytab("admin@EXAMPLE.COM", "/etc/security/keytabs/admin.keytab");
		DriverManagerDataSource dataSource = new DriverManagerDataSource(
				"jdbc:hive2://10.60.2.26:10001/default;principal=hive/example-qa1-dn2@EXAMPLE.com", "hive", "");
		dataSource.setDriverClassName("org.apache.hive.jdbc.HiveDriver");
		JdbcTemplate jdbcTemplate = new JdbcTemplate();
		jdbcTemplate.setDataSource(dataSource);
		long count = 0;
		count = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM provider", Long.class);
		System.out.println("Count is *************************" + count);
	}
}

I have submitted workflow job using admin user who have access to hive.

I did kinit programmatically as well using UserGroupInformation API for admin user, but still I am getting above exception.

I have kept only spring core, spring jdbc and hive jdbc jars in oozie HDFS cache.

I also tried all possible solutions mentioned on hortonworks(https://community.hortonworks.com/questions/8995/access-to-hive-from-oozie-java-action-with-kerbero.html), stack overflow and other forums, however I am still facing same issue.

Can anyone suggest any solution for this issue?

5 REPLIES 5

Re: Executing Hive queries through oozie java action on kerberized cluster

So I do not see the JavaAction in the list of supported Actions, so not sure if it should work. You may have to upload the keytab file with the <file> tag in oozie and then do the kinit in your java application.

https://oozie.apache.org/docs/4.2.0/DG_ActionAuthentication.html

http://stackoverflow.com/questions/33211134/hbase-kerberos-connection-renewal-strategy/

  • The Java Way: on startup, before you connect to HBase/HDFS/whatever, create explicitly an UGI with loginUserFromKeytab() then run checkTGTAndReloginFromKeytab() from time to time

Re: Executing Hive queries through oozie java action on kerberized cluster

Expert Contributor

jdbc:hive2://10.60.2.26:10000/default;principal=hive/hscale-qa1-dn2@HSCALE.com;auth=NOSASL

Shouldn't this be

auth=kerberos

Re: Executing Hive queries through oozie java action on kerberized cluster

New Contributor

I tried with auth=kerberos, but it didn't work.

Re: Executing Hive queries through oozie java action on kerberized cluster

Expert Contributor

@Ashish Gupta - I have similar problem as well - while kinit before running the process works I cannot get the process to work when logging in to kerberos programmatically. I was told this is an issue with some spark version but obviously you are not using spark...

Re: Executing Hive queries through oozie java action on kerberized cluster

New Contributor