04-01-2019 07:16 PM
Hi Cloudera Community,
Why Cloudera don't recommended the creation of root.[username] pool on the Resource Pools configuration ? I can't find docummentation realted with this.
04-05-2019 08:46 AM
Our recommendation is to place [username] pools under a parent pool, such as root.users.[username]. This allows you to control overall usage of [username] queues relative to other pools under the root. This way, if you have 3 root level pools:
...you can define appropriate weights at amonst these 3 pools.
If you configured root.[username], each user pool will be added with a default share of 1. So, for example, if your initial configuration was:
... root.[username] placement rule
In the beginning, root.production and root.adhoc will each have 50% of cluster resources. When the first user runs a job, their subpool is created with a default weight of 1.
Now imagine that 50 users run a job and 50 new pools are created at the root level with weight of 1. All of the sudden you have weighted resources heavily to the user pools:
50 root.[username] generated pools (with 50 users)
So in summary, you may prefer to do something that would limit all users, no matter how many, to a fixed ratio of cluster resources:
-> root.users.[username] placement rule