Reply
New Contributor
Posts: 1
Registered: ‎11-09-2015

Unable to run java action through oozie in kerberized cluster

Unable to run java action through oozie in kerberized cluster.  We get the following errors:

 

 

 

 Log Type: stderr
 
  Log Upload Time: Tue Dec 08 10:16:48 -0500 2015
 
  Log Length: 6367
  java.sql.SQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500168) Unable to connect to server: GSS initiate failed.
  at com.cloudera.hive.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
  at com.cloudera.hive.hivecommon.api.ZooKeeperEnabledExtendedHS2Factory.createClient(Unknown Source)
  at com.cloudera.hive.hivecommon.core.HiveJDBCCommonConnection.connect(Unknown Source)
  at com.cloudera.hive.hive.core.HiveJDBCConnection.connect(Unknown Source)
  at com.cloudera.hive.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
  at com.cloudera.hive.jdbc.common.AbstractDriver.connect(Unknown Source)
  at java.sql.DriverManager.getConnection(DriverManager.java:571)
  at java.sql.DriverManager.getConnection(DriverManager.java:233)
  at converter.jdbctest.main(jdbctest.java:161)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:57)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
  at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:38)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  Caused by: com.cloudera.hive.support.exceptions.GeneralException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500168) Unable to connect to server: GSS initiate failed.
  ... 32 more
  Caused by: java.lang.RuntimeException: [Cloudera][HiveJDBCDriver](500168) Unable to connect to server: GSS initiate failed
  at com.cloudera.hive.hivecommon.api.HiveServerPrivilegedAction.run(Unknown Source)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:356)
  at com.cloudera.hive.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
  at com.cloudera.hive.hivecommon.api.ZooKeeperEnabledExtendedHS2Factory.createClient(Unknown Source)
  at com.cloudera.hive.hivecommon.core.HiveJDBCCommonConnection.connect(Unknown Source)
  at com.cloudera.hive.hive.core.HiveJDBCConnection.connect(Unknown Source)
  at com.cloudera.hive.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
  at com.cloudera.hive.jdbc.common.AbstractDriver.connect(Unknown Source)
  at java.sql.DriverManager.getConnection(DriverManager.java:571)
  at java.sql.DriverManager.getConnection(DriverManager.java:233)
  at converter.jdbctest.main(jdbctest.java:161)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:57)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
  at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:38)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
  Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
  at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
  at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
  at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
  ... 36 more
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Posts: 1,896
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: Unable to run java action through oozie in kerberized cluster

Have you followed the Oozie Java Action documentation, specifically the two sections copy-quoted below?

"""
A java action can create a Hadoop configuration for interacting with a cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop configuration file which includes the environments site configuration files (e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added to the section of the java action. The Hadoop configuration file is made available as a local file to the Java application in its running directory. It can be added to the java actions Hadoop configuration by referencing the system property: oozie-action.conf.xml . For example:

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///", System.getProperty("oozie.action.conf.xml")));
"""

And,

"""
// propagate delegation related props from launcher job to MR job
if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
jobConf.set("mapreduce.job.credentials.binary", System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
}
"""

X-Ref: http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

Also, the Cloudera JDBC Connector for Hive may not support token based authentication, and was not intended for use via Oozie (unless you can do a kerberos auth from your Java action using a passed keytab directly).

Is there a reason you cannot use the regular Hive JDBC driver for this purpose instead? It does support delegation token based auth, like the Hive2 action in Oozie supports: http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_Hive2ActionExtension.html
Announcements