- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Integrate Hive on Spark with Hue Cloudera CDP 7.1.7
- Labels:
-
Apache Hive
-
Apache Spark
-
Apache YARN
Created 09-20-2023 11:27 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello everyone,
in a cdp 7.1.7 cluster I have not found a way to implement engine spark to run hive queries on hue. I can execute hive queries with spark engine with spark-shell but if I try to execute hive queries on hue by setting set hive.execution.engine=spark I get the following error:
Error while compiling statement: FAILED: Execution Error, return code 30041 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Failed to create Spark client for Spark session
Can anyone help me to implement hive on spark via hue?
Thanks in advance
Created 09-21-2023 06:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hive is unaware of Spark and won't use it to execute commands
On the other hand, Spark can access Hive data using the HiveWarehouseConnector
Hope this helps,
-JMP
Created 09-20-2023 11:47 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Lorenzo Welcome to the Cloudera Community!
To help you get the best possible solution, I have tagged our Hive experts @Shmoo @mszurap who may be able to assist you further.
Please keep us updated on your post, and we hope you find a satisfactory solution to your query.
Regards,
Diana Torres,Community Moderator
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Created 09-20-2023 11:53 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Lorenzo On CDP Hive the Spark engine is deprecated if favor of TEZ
Created 09-20-2023 11:55 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
And there is no way currently to integrate spark?
Created 09-21-2023 06:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hive is unaware of Spark and won't use it to execute commands
On the other hand, Spark can access Hive data using the HiveWarehouseConnector
Hope this helps,
-JMP
Created 09-22-2023 06:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you, for now i implemented scala on hue and it works
