Support Questions

Find answers, ask questions, and share your expertise

How to load Hive data in to Spark-shell

avatar

I have downloaded Cloudera quickstart 5.10 for VirtualBox.

But it's not loading hive data into spark 

 

import org.apache.spark.sql.hive.HiveContext
import sqlContext.implicits._
val hiveObj = new HiveContext(sc)

hiveObj.refreshTable("db.table") // if you have uograded your hive do this, to refresh the tables.

val sample = sqlContext.sql("select * from table").collect()
sample.foreach(println)

 

Still i'm getting the error as table not found(It's not accessing metadata)

What should i do, Any one pls help me

(In cloudera quickstart we are unable to copy hive-site.xml in to spark/conf)

 

14 REPLIES 14

avatar
Champion

 now thats the reason it says table not found . mate 

Will dig more and comeback to u  . we almost narrow down to it 

Could you see if you have hive-site.xml & hdfs-site.xml in your spark conf folder 

 

/etc/spark/conf/

if not just do cp command and push those xml file  to /etc/spark/conf/ restart spark 

fire it again let us see . 

 

 

 

avatar

Thank you Guna.

i link the confuguration file hive to spark as

ln -s /etc/hive/conf/hive-site.xml /etc/spark/conf/hive-site.xml

it started workig after restart only.

 

avatar
Champion

Hurray ! 🙂 @hadoopSparkZen

avatar
New Contributor

@csgunaand @hadoopSparkZen  you guys have saved my day. Thanks to both of you 🙂

avatar
New Contributor

Hi, I am also facing the same issue. Not able to load hive table into spark.

 

I tried to copy the xml files in spark conf folder. But its permission denied and I tries to change the permission for the folder also .That is also not working.

 

Using cloudera vm 5.12