I have cloudera setuo with following version's
Hive release with parcels -1.1.0-cdh5.15.0
I want to run hive queries in my spark application.
In scala it would be like
val df = sqlContext.sql("Select * from default.test")
. However it alwasy gives me error of database ot view not found, which actually exists.
I added hive lib directory to spark.driver.extraClassPath in spark_defaults.conf. But still same results.
I tried copying all hive jar's to spark2/jars/ folder but still same results
Hive Execution engine is MR.
Do i need to add any extra jar or classpath to spark configurations.?
How will spark pick hive jar's. Please help me out on this
Thanks for your reply. I already copied it but it seems we need to choose hive as option in hive-service configurations in cloudera manager.
However when i choose hive it shows all the databases in hive.
Also spark-defaults.conf is got updated with hive-metastore version and hive jars.
But i got a new error of
This error seems incompatible hive.serde jar .
What's the workaround for it ?