Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is description of YARN Scheduler Queue Mapping Override property wrong in HDP Docs?

Highlighted

Is description of YARN Scheduler Queue Mapping Override property wrong in HDP Docs?

Expert Contributor

After checking the official Apache Hadoop docs and some other Internet sources it seems to me that the description of the behavior for the property

yarn.scheduler.capacity.queue-mappings-override.enable=true/false

of the YARN Capacity Scheduler is wrong (and actually the opposite) in the official documentation of HDP 3.1.0 at the "Managing Data Operating System | Enable Override of Default Queue Mappings" section.

It states that when this property is set to true "it allow the user to override the default user to queue mappings defined with the yarn.scheduler.capacity.queue-mappings property" , and indeed gives an example showing this (wrongly) expected behavior.

However, as clearly explained in the responses by @gnovak  and @vasubramanian  for the question "Yarn queue Mapping" in this same forum, the purpose of that property is exactly the opposite:

  • if yarn.scheduler.capacity.queue-mappings-override.enable is set to false then user's have more control on which queue they want to submit the application either by setting the queue name or from default mapping, in this case user specified queue name takes the precedence , but if yarn.scheduler.capacity.queue-mappings-override.enable set to true then admin's have more control and admin can override user's explicit settings.

In other words, what the property enables is the possibility for the administrator to use the property "yarn.scheduler.capacity.queue-mappings" to override what the user or application configuration states, and not the other way.

 

May please some of the most experienced contributors in this forum confirm this? and if that's the case, if it were possible that someone from the company could correct such and important error in the documentation?

Best regards.