question Integrate Hive on Spark with Hue Cloudera CDP 7.1.7 in Support Questions

Integrate Hive on Spark with Hue Cloudera CDP 7.1.7

Lorenzo — Wed, 20 Sep 2023 18:27:01 GMT

Hello everyone,
in a cdp 7.1.7 cluster I have not found a way to implement engine spark to run hive queries on hue. I can execute hive queries with spark engine with spark-shell but if I try to execute hive queries on hue by setting set hive.execution.engine=spark I get the following error:
Error while compiling statement: FAILED: Execution Error, return code 30041 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Failed to create Spark client for Spark session

Can anyone help me to implement hive on spark via hue?

Thanks in advance

Re: Integrate Hive on Spark with Hue Cloudera CDP 7.1.7

DianaTorres — Wed, 20 Sep 2023 18:47:58 GMT

@Lorenzo Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Hive experts @Shmoo @mszurap who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Re: Integrate Hive on Spark with Hue Cloudera CDP 7.1.7

JoseManuel — Wed, 20 Sep 2023 18:53:32 GMT

@Lorenzo On CDP Hive the Spark engine is deprecated if favor of TEZ

Re: Integrate Hive on Spark with Hue Cloudera CDP 7.1.7

Lorenzo — Wed, 20 Sep 2023 18:55:00 GMT

And there is no way currently to integrate spark?

Re: Integrate Hive on Spark with Hue Cloudera CDP 7.1.7

JoseManuel — Thu, 21 Sep 2023 13:52:15 GMT

@Lorenzo

Hive is unaware of Spark and won't use it to execute commands

On the other hand, Spark can access Hive data using the HiveWarehouseConnector

Hope this helps,

-JMP

Re: Integrate Hive on Spark with Hue Cloudera CDP 7.1.7

Lorenzo — Fri, 22 Sep 2023 13:02:09 GMT

Thank you, for now i implemented scala on hue and it works