05-21-2014 02:11 AM
and why there is no capacity-scheduler.xml in yarn's conf folder? as described in http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
05-21-2014 07:21 AM
See documentation here:
Dynamic Resource Pools uses the Fair Scheduler, not the Capacity Scheduler. So you don't need capacity-scheduler.xml.
Also keep in mind that cloudera doesn't generally put server-only configuration into /etc/hadoop/conf. You can view a particular role's configuration files in the CM UI on the Processes tab for that role (click Show to unhide them). These will appear in a directory in /var/run/cloudera-scm-agent/. You can see the directory name in the stderr log on the Processes tab.
05-22-2014 02:59 AM
thanks, i found the config files that are changed
they are in
on yarn RM node
on impalad node
03-07-2019 09:10 AM - edited 03-07-2019 09:10 AM
Is it possible, an YARN DRP has priority against other pools?
ie. If i already have 2 DRP created
DRP1 = 500Gb
DRP2 = 200Gb
and i want to add new one DRP3 = 700Gb that have priority against other pools and use all available resources are directed to DRP3. Is this possible?
04-09-2019 05:54 AM
You can try to increase the weight of the DRP3, so it will get more priority and jobs submitted to this pool will get more resources than other pools based on the weight configured.