Skip to main content
vaibhavgokhale
Silver ⭐️
Jul 3, 2023 1:24 PM UTC
Solved

Configuring Tez queue for Hive Warehouse Connector

Hi,

 

I am running a query on Hive through my Spark application using HiveWarehouseConnector.

I want to use a particular YARN queue for Tez job launched by HiveWarehouseConnector (custom queue configuration at application level).

I have tried following two ways:

1. By using Spark conf and setting spark.hive.tez.queue.name = <queue name>

2. By setting tez.queue.name parameter in the hiveserver2 URL as suggested in the following thread https://community.cloudera.com/t5/Support-Questions/Setting-yarn-queue-for-hive-with-beeline/td-p/161499

 

I am able to set the queue for beeline using the URL option.

 

However, any of the options is not working for HiveWarehouseConnector.

 

Can anyone please help in this regard?

 

smruti
smrutiAnswer
Gold ⭐️⭐️⭐️
Jul 3, 2023 8:35 PM UTC

@vaibhavgokhale You could try:

--conf spark.sql.hive.conf.list="tez.queue.name=queue1"

 

3 replies

DianaTorres
Community Manager
Community Manager
Jul 3, 2023 4:06 PM UTC

 @vaibhavgokhale Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Tez experts @MGreen and @Shmoo  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
smruti
Gold ⭐️⭐️⭐️
smrutiAnswer
Gold ⭐️⭐️⭐️
Jul 3, 2023 8:35 PM UTC

@vaibhavgokhale You could try:

--conf spark.sql.hive.conf.list="tez.queue.name=queue1"

 

Translate
vaibhavgokhale
Silver ⭐️
Jul 4, 2023 11:10 AM UTC

Thanks @smruti for quick response. This is working.

Translate