Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Issue with Spark on Hive.

avatar
Contributor

I can run Queries using hive with the map reduce engine, but when running hive on spark it fails giving an expection:

Failed to execute spark task, with exception: 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)'

1 ACCEPTED SOLUTION

avatar

We don't support Spark on Hive, try Hive on Tez.

We do plan to support SparkSQL when Spark 1.5.1 becomes GA on HDP.

View solution in original post

9 REPLIES 9

avatar

What is your HDP, Spark and Hive version?

avatar
Master Mentor

avatar
Master Mentor
15/11/03 06:56:07 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.NoSuchFieldError: SPARK_RPC_CLIENT_CONNECT_TIMEOUT
java.lang.NoSuchFieldError: SPARK_RPC_CLIENT_CONNECT_TIMEOUT
	

15/11/03 06:56:17 INFO util.Utils: Shutdown hook called

avatar
Master Mentor

@visakh.nair@hortonworks.com

Please see this "Work in progress"

avatar

We don't support Spark on Hive, try Hive on Tez.

We do plan to support SparkSQL when Spark 1.5.1 becomes GA on HDP.

avatar
New Contributor

Do you have any update on this issue : 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)'

avatar
New Contributor

Do you know whether hive on spark supported in HDP2.4 and above with Hive as 1.2

avatar

We still don't support Hive on Spark. We support Hive on Tez.

We also support SparkSQL.

avatar

I can indeed as vshukla is saying recommend using SparkSQL with the HiveSqlContext.