Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Cloudera Oozie workflow spark JDBC access to hive GSS initiate failed

avatar
author-rank saikat
Explorer

Created on ‎05-12-2021 05:50 AM - last edited on ‎04-21-2026 02:04 AM by Community Manager Community Manager

I am trying to connect Hive from spark with JDBC connection. While I use spark-submit option from terminal, it's completed successfully. But from Oozie workflow it's showing ERROR  Unable to read HiveServer2 configs from ZooKeeper

 

Here is my code snippet

 

 

val sparkSession: SparkSession = SparkSession
     .builder().appName("Spark hive" )
     .getOrCreate()

   val jdbcUrl = "jdbc:hive2://master1.cloudera.com:2181," +
     "master2.cloudera.com:2181,master3.cloudera.com:2181/default;" +
     "serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"

  val driverClass = "org.apache.hive.jdbc.HiveDriver"
  try {
    val cls = Class.forName(driverClass)
    println(cls.getName)
  } catch {
    case e: Throwable => e.printStackTrace()
  }

  val connection = DriverManager.getConnection(jdbcUrl, null,null)
  println(connection.isClosed)

  val statement: Statement = connection.createStatement()
  val table = "default.external_tab"
  val sql = "SELECT * FROM " + table
  val result = statement.executeQuery(sql)
  println("Running: " + sql)
  while ( {
    result.next
  }) {
    println("Name=" + result.getString(1))
  }
  result.close()
  statement.close()
  connection.close()

 

 

Spark submit command

 

spark-submit \
--master yarn \
--deploy-mode client \
--principal [email protected] \
--keytab /var/tmp/saikat.keytab \
--class com.example.DataLoadMain \
spark-hive-load_2.11-0.1.jar

 

 

Oozie workflow job xml

 

<workflow-app name="Spark Workflow" xmlns="uri:oozie:workflow:0.5">
    <start to="spark-cd37"/>
    <kill name="Kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <action name="spark-b291">
        <spark xmlns="uri:oozie:spark-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <master>yarn</master>
            <mode>client</mode>
            <name></name>
              <class>com.example.DataLoadMain</class>
            <jar>spark-hive-load_2.11-0.1.jar</jar>
              <spark-opts>--master yarn --deploy-mode client --keytab /var/tmp/saikat.keytab --principal [email protected] --conf spark.yarn.appMasterEnv.HADOOP_JAAS_DEBUG=true --conf spark.yarn.appMasterEnv.SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark/ --py-files hive-jdbc-3.1.3000.7.1.3.0-100.jar</spark-opts>
            <file>/user/saikat/spark-hive-load_2.11-0.1.jar#spark-hive-load_2.11-0.1.jar</file>
        </spark>
        <ok to="End"/>
        <error to="Kill"/>
    </action>
    <end name="End"/>
</workflow-app>

 

 

Oozie spark action error log

 

org.apache.hive.jdbc.HiveDriver
17:49:58.240 [main-EventThread] ERROR org.apache.curator.framework.imps.EnsembleTracker - Invalid config event received: {server.1=master3.cloudera.com:3181:4181:participant, version=0, server.3=master1.cloudera.com:3181:4181:participant, server.2=master2.cloudera.com:3181:4181:participant}
17:49:58.251 [main-EventThread] ERROR org.apache.curator.framework.imps.EnsembleTracker - Invalid config event received: {server.1=master3.cloudera.com:3181:4181:participant, version=0, server.3=master1.cloudera.com:3181:4181:participant, server.2=master2.cloudera.com:3181:4181:participant}
17:49:58.401 [main] ERROR org.apache.thrift.transport.TSaslTransport - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
    at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) ~[?:1.8.0_251]
    at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) [hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:51) [hive-standalone-metastore-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:48) [hive-standalone-metastore-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_251]
    at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_251]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) [hadoop-common.jar:?]
    at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport.open(TUGIAssumingTransport.java:48) [hive-standalone-metastore-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:441) [hive-jdbc-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:317) [hive-jdbc-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) [hive-jdbc-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
    at java.sql.DriverManager.getConnection(DriverManager.java:664) [?:1.8.0_251]
    at java.sql.DriverManager.getConnection(DriverManager.java:247) [?:1.8.0_251]
    at com.example.DataLoadMain$.delayedEndpoint$com$example$DataLoadMain$1(DataLoadMain.scala:52) [spark-hive-load_2.11-0.1.jar:0.1]
    at com.example.DataLoadMain$delayedInit$body.apply(DataLoadMain.scala:11) [spark-hive-load_2.11-0.1.jar:0.1]
    at scala.Function0$class.apply$mcV$sp(Function0.scala:34) [scala-library-2.11.12.jar:?]
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) [scala-library-2.11.12.jar:?]
    at scala.App$$anonfun$main$1.apply(App.scala:76) [scala-library-2.11.12.jar:?]
    at scala.App$$anonfun$main$1.apply(App.scala:76) [scala-library-2.11.12.jar:?]
    at scala.collection.immutable.List.foreach(List.scala:392) [scala-library-2.11.12.jar:?]
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35) [scala-library-2.11.12.jar:?]
    at scala.App$class.main(App.scala:76) [scala-library-2.11.12.jar:?]
    at com.example.DataLoadMain$.main(DataLoadMain.scala:11) [spark-hive-load_2.11-0.1.jar:0.1]
    at com.example.DataLoadMain.main(DataLoadMain.scala) [spark-hive-load_2.11-0.1.jar:0.1]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_251]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_251]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_251]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_251]
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:847) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:922) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:931) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) [spark-core_2.11-2.4.0.7.1.3.0-100.jar:2.4.0.7.1.3.0-100]
    at org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:187) [oozie-sharelib-spark-5.1.0.7.1.3.0-100.jar:?]
    at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:94) [oozie-sharelib-spark-5.1.0.7.1.3.0-100.jar:?]
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107) [oozie-sharelib-oozie-5.1.0.7.1.3.0-100.jar:?]
    at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:61) [oozie-sharelib-spark-5.1.0.7.1.3.0-100.jar:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_251]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_251]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_251]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_251]
    at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413) [oozie-sharelib-oozie-5.1.0.7.1.3.0-100.jar:?]
    at org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55) [oozie-sharelib-oozie-5.1.0.7.1.3.0-100.jar:?]
    at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226) [oozie-sharelib-oozie-5.1.0.7.1.3.0-100.jar:?]
    at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_251]
    at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_251]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) [hadoop-common.jar:?]
    at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220) [oozie-sharelib-oozie-5.1.0.7.1.3.0-100.jar:?]
    at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) [oozie-sharelib-oozie-5.1.0.7.1.3.0-100.jar:?]
    at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_251]
    at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_251]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) [hadoop-common.jar:?]
    at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) [oozie-sharelib-oozie-5.1.0.7.1.3.0-100.jar:?]
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
    at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:148) ~[?:1.8.0_251]
    at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) ~[?:1.8.0_251]
    at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:189) ~[?:1.8.0_251]
    at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) ~[?:1.8.0_251]
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) ~[?:1.8.0_251]
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.8.0_251]
    at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ~[?:1.8.0_251]
    ... 57 more
17:49:58.418 [main-EventThread] ERROR org.apache.curator.framework.imps.EnsembleTracker - Invalid config event received: {server.1=master3.cloudera.com:3181:4181:participant, version=0, server.3=master1.cloudera.com:3181:4181:participant, server.2=master2.cloudera.com:3181:4181:participant}
17:49:58.418 [main-EventThread] ERROR org.apache.curator.framework.imps.EnsembleTracker - Invalid config event received: {server.1=master3.cloudera.com:3181:4181:participant, version=0, server.3=master1.cloudera.com:3181:4181:participant, server.2=master2.cloudera.com:3181:4181:participant}
17:49:58.521 [main] ERROR org.apache.hive.jdbc.Utils - Unable to read HiveServer2 configs from ZooKeeper

 

2,880 Views
0 Kudos
2 REPLIES 2

avatar
author-rank Daming Xue author-rank
Expert Contributor

Created ‎05-13-2021 07:42 PM

Hello

 

Can you review this post and check the whether Hive namespace is created in ZK?

 

https://community.cloudera.com/t5/Support-Questions/How-to-resolve-Unable-to-read-HiveServer2-config...

2,854 Views
0 Kudos

avatar
author-rank saikat
Explorer

Created ‎05-15-2021 10:29 PM

@Daming Xue Namespace is created.

[serverUri=node6.cloudera.com:10000;version=3.1.3000.7.1.3.0-100;sequence=0000000018]
2,839 Views
0 Kudos
Announcements
Community Announcements
Community Upgrade Announcement
What's New @ Cloudera
Announcing Cloudera Flow Management Operator for Kubernetes ...
What's New @ Cloudera
Announcing Cloudera Streams Messaging Operator for Kubernete...
Community Announcements
July 2026 Community Highlights
Community Announcements
June 2026 Community Highlights
View More Announcements