Created 11-16-2016 11:52 AM
Simple sparkSQL application works: (In CDH 5.8.0)
Oozie exception:
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
Spark Code:----------------------------
import org.apache.spark._
import org.apache.spark.sql._
import org.apache.spark.sql.hive._
object SQLAction{
def main(args:Array[String])
{
val conf = new SparkConf()
val sc = new SparkContext(conf)
val sqlc = new HiveContext(sc) // cause exception
println("sparkSQL success") //not get run
sqlc sql "select * from testcase_pos" collect
println("select success")
sc.stop
}
}
----------------------------
What we tried:
Created 11-17-2016 03:53 PM
Hi maybe try uploading your hive-site.xml to hdfs and referancing it in the oozie options the gearwheel on the cell.
in the Options box something like this
--files ${nameNode}/Analytics/Spark_jars/hive-site.xml
Thanks