Hive Context in CDH 5.3.x

[#1]

We have just started with cloudera. I have a question about spark. Does CDH 5.3.x supports Hive Context?

I have installed CDH 5.3. When I try to get the hive context on scala editor. I get the following error

val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)

error: bad symbolic reference. A signature in HiveContext.class refers to term hive in package org.apache.hadoop which is not available. It may be completely missing from the current classpath, or the version on the classpath might be incompatible with the version used when compiling HiveContext.class. error:

Thanks,

Satya K

[#2]

The simplest first check is to ensure the node is a Hive Gateway. You can
set this role in Cloudera Manager.

[#3]

Hello Gautam:

Thanks for your reply. I have seen the status, actually hive gateway is added on all the nodes, but it is not getting started on any of the nodes and gateway role status and health is always unknown. Are there any other steps I need to do to get this service up? there were no logs for this service as well

While I was installing the CDH 5.3, all the health checks were passed. One other thing is I have let the installer select all the roles. I didn't select any. Please let me know how can I proceed. Thanks much for your help

Thanks,

Satya K 

[#4]

Also I tried to restart the complete hive service and also this specific role. When I stop this gateway role for all the nodes, it's stopped successfully, but when try to start it again it says "Service must be stopped.", but the service is already stopped. I was trying to search any other blogs/ cloudera community to get a solution. Coudln't find a solution.

Thanks,

Satya Kota

[#5]

The gateway role isn't something that is started. Setting a gateway role
simply copies hadoop configuration files under /etc//conf. So Hive
gateway means /etc/hive/conf. For YARN it would be /etc/yarn/conf and so on.

I'll let someone else respond about the Spark questions.

[#6]

Thanks Gautam, Got it.

Hello Guys,

Any one else tried using the hive context in spark installed with CDH 5.3.0? Can you please provide your input on this?

Thanks,

Satya K

[#7]

Tried few different ways I don't think it works in CDH which is wiered. I have installed plain apcahe spark on a fresh cluster, it all works normal

[#8]

Spark with CDH 5.3.0 doesn't officially support Hive afaik. Glad you
got it working with plain Apache Spark.

[#9]

Thanks Gautam for the information. Are there plans for supporting this in future versions?

Does this mean with CDH 5.3.0 as of today we won't get spark-sql on hive, which is a known thing?

Thanks,

Satya K

[#10 · Accepted solution]

Hive on Spark in CDH 5.3.0 is not supported but i am not privy to any
timetables. Sorry I can't help in that way. I would suggest that you
subscribe to the Announcements forum so you are updated when a new
version is released.

http://community.cloudera.com/t5/Release-Announcements/bd-p/RelAnnounce