Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Cloudera 7.4.4 - Yarn - Questions about Queues

avatar
Explorer

Hello.

In Yarn Queue Manager, there is 4 queues defined :

2024-02-08 09_21_12-Cloudera Manager.png

but when we submit queries (using Hue or SAS Software to load data), only the default queue is used.

And we can go very high :

2024-02-08 09_14_50-YARN.png

From what we undestand, Default queue is defined for using 20% of the cluster but can use up to 100%, so 70% is normal.

2024-02-08 09_15_58-YARN.png

But our cluster is 365 GB / 15 vCores and some applications can use more :

2024-02-08 09_17_27-YARN.png

7 / 7 / 6 / 5 / 3 vCores in the graph = 28 vCore !

How is it possible ?

Thanks for your help

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor

The application ask for container run some part in that container and then release it back.
So the 28 vcores that you are seeing is due to that.

Let's say your job asks for 4 containers and eack with 7 vcores so at first only two containers will run as you have limit of 15 vcores. but if one container is released then that job will take another container with 7 vcores so in total now the number of vcores used is 21.

View solution in original post

1 REPLY 1

avatar
Expert Contributor

The application ask for container run some part in that container and then release it back.
So the 28 vcores that you are seeing is due to that.

Let's say your job asks for 4 containers and eack with 7 vcores so at first only two containers will run as you have limit of 15 vcores. but if one container is released then that job will take another container with 7 vcores so in total now the number of vcores used is 21.