I am unable to configure spark as a execution engine in hive. Could you please help me with this
i am using HDP 2.4.
Spark is not yet a GA feature in Hive, still very much in dev phase. You can however use SparkSQL to issue queries in a hive context to use Hive tables.
View solution in original post
Use hive.execution.engine=tez for better performance
When can we expect spark to be GA with HIVE?
Can you please provide logs or error messages which you are getting?
Also did you tried using below parameters on hive shell?
We support MR or Tez as the execution engine for Hive. We also support SparkSQL.
By setting the above parameters on hive shell,
it is giving correct output for simple select * from table name query,
but is giving error as
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
when we do select count(*) from table name query.
I have also added spark -assembly-hadoop jar in hive/lib
please guide me ,i am new to spark