Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive Context in CDH 5.3.x

avatar
Explorer

We have just started with cloudera. I have a question about spark. Does CDH 5.3.x supports Hive Context?

 

I have installed CDH 5.3. When I try to get the hive context on scala editor. I get the following error

 

val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)

 

error: bad symbolic reference. A signature in HiveContext.class refers to term hive in package org.apache.hadoop which is not available. It may be completely missing from the current classpath, or the version on the classpath might be incompatible with the version used when compiling HiveContext.class. error:

 

Thanks,

Satya K

 

1 ACCEPTED SOLUTION

avatar
Hive on Spark in CDH 5.3.0 is not supported but i am not privy to any
timetables. Sorry I can't help in that way. I would suggest that you
subscribe to the Announcements forum so you are updated when a new
version is released.

http://community.cloudera.com/t5/Release-Announcements/bd-p/RelAnnounce

Regards,
Gautam Gopalakrishnan

View solution in original post

14 REPLIES 14

avatar
Explorer

Ok. Thank you !

avatar
Master Collaborator

Not supported, but, all of the standard bits are there. It should work just like any other installation. You will probably have to put the Hive jars on your classpath manually, says Marcelo.

avatar
Expert Contributor

can you please tell me how to add the hive jars in the classpath ?

avatar
Expert Contributor

avatar
Expert Contributor

okey i will try this thanks so much , but i have another question if you please, i add the external jars i need , all of them work normally , but the 

" org.apache.hadoop.hive.conf.HiveConf" which exists in hive-common-0.13.1-cdh5.3.0.jar  , it gives the error " calss not found " so why that happens ?

 

the command i run :

 

sudo spark-submit --class "WordCount" --master local[*] --jars /usr/local/WordCount/target/scala-2.10/spark-streaming-flume_2.11-1.2.0.jar,/usr/lib/avro/avro-ipc-1.7.6-cdh5.3.0.jar,/usr/lib/flume-ng/lib/flume-ng-sdk-1.5.0-cdh5.3.0.jar,/usr/lib/hive/lib/hive-common-0.13.1-cdh5.3.0.jar,/usr/local/WordCount/target/scala-2.10/spark-hive_2.10-1.2.0-cdh5.3.0.jar /usr/local/WordCount/target/scala-2.10/wordcount_2.10-1.0.jar 127.0.0.1 9999