Issue with Spark on Hive.

[#1]

I can run Queries using hive with the map reduce engine, but when running hive on spark it fails giving an expection:

Failed to execute spark task, with exception: 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)'

[#2]

What is your HDP, Spark and Hive version?

[#3]

@visakh.nair@hortonworks.com

I am able to reproduce this

HDP 2.3.2

https://issues.apache.org/jira/browse/HIVE-9970

and https://issues.apache.org/jira/browse/HIVE-7980

I am working on finding the fix

[#4]

15/11/03 06:56:07 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.NoSuchFieldError: SPARK_RPC_CLIENT_CONNECT_TIMEOUT
java.lang.NoSuchFieldError: SPARK_RPC_CLIENT_CONNECT_TIMEOUT
	

15/11/03 06:56:17 INFO util.Utils: Shutdown hook called

[#5]

@visakh.nair@hortonworks.com

Please see this "Work in progress"

[#6 · Accepted solution]

We don't support Spark on Hive, try Hive on Tez.

We do plan to support SparkSQL when Spark 1.5.1 becomes GA on HDP.

[#7]

Do you have any update on this issue : 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)'

[#8]

Do you know whether hive on spark supported in HDP2.4 and above with Hive as 1.2

[#9]

We still don't support Hive on Spark. We support Hive on Tez.

We also support SparkSQL.

[#10]

I can indeed as vshukla is saying recommend using SparkSQL with the HiveSqlContext.