Skip to main content
HadoopHero
Silver ⭐️⭐️⭐️
June 13, 2023
Solved

Time rules on capacity scheduler of YARN

Hi,

I'm using the Cloudera CDP 7.1.7 Private Cloud Base version and would like to have a confirmation if there is a possibility to be able to set, in the capacity scheduler of Yarn, time rules as I did previously in the old CDH.
It would be interesting to be able to use that functionality also in CDP, but in the Cloudera documentation I've searched on the web there's no mention of it.

Could you give me some hints if this feature is still present in CDP?

HadoopHero_0-1686690996421.png

 

Thanks

 

Yuexin Zhang
Gold ⭐️
June 14, 2023

Hi

There's similar feature in CDP as the time based scheduling rules in CDH, called "Dynamic Queue Scheduling". But it is Tech Preview in CDP 717 and GA in CDP 718, please refer to below docs:

https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/yarn-allocate-resources/topics/yarn-dynamic-queue-scheduling.html

 

https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/yarn-allocate-resources/topics/yarn-dynamic-queue-scheduling.html

 

Hope this helps.

3 replies

DianaTorres
Community Manager
Community Manager
June 14, 2023

@HadoopHero Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Yarn experts @Bharati  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
Yuexin Zhang
Gold ⭐️
Gold ⭐️
June 14, 2023

Hi

There's similar feature in CDP as the time based scheduling rules in CDH, called "Dynamic Queue Scheduling". But it is Tech Preview in CDP 717 and GA in CDP 718, please refer to below docs:

https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/yarn-allocate-resources/topics/yarn-dynamic-queue-scheduling.html

 

https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/yarn-allocate-resources/topics/yarn-dynamic-queue-scheduling.html

 

Hope this helps.

Translate
HadoopHero
Silver ⭐️⭐️⭐️
June 14, 2023

Hi Yuexin,

you have been very helpful. Unfortunately, if I wanted to use "Dynamic Queue Scheduling" in CDP 717 at the moment, I would not have the guarantee to solve any problems via Cloudera support. In fact, it is not recommended to use it in production.

Thank you very much

Translate