Support Questions
Find answers, ask questions, and share your expertise

Is description of YARN Scheduler Queue Mapping Override property wrong in HDP Docs?

Highlighted

Is description of YARN Scheduler Queue Mapping Override property wrong in HDP Docs?

Expert Contributor

After checking the official Apache Hadoop docs and some other Internet sources it seems to me that the description of the behavior for the property

yarn.scheduler.capacity.queue-mappings-override.enable=true/false

of the YARN Capacity Scheduler is wrong (and actually the opposite) in the official documentation of HDP 3.1.0 at the "Managing Data Operating System | Enable Override of Default Queue Mappings" section.

It states that when this property is set to true "it allow the user to override the default user to queue mappings defined with the yarn.scheduler.capacity.queue-mappings property" , and indeed gives an example showing this (wrongly) expected behavior.

However, as clearly explained in the responses by @gnovak  and @vasubramanian  for the question "Yarn queue Mapping" in this same forum, the purpose of that property is exactly the opposite:

  • if yarn.scheduler.capacity.queue-mappings-override.enable is set to false then user's have more control on which queue they want to submit the application either by setting the queue name or from default mapping, in this case user specified queue name takes the precedence , but if yarn.scheduler.capacity.queue-mappings-override.enable set to true then admin's have more control and admin can override user's explicit settings.

In other words, what the property enables is the possibility for the administrator to use the property "yarn.scheduler.capacity.queue-mappings" to override what the user or application configuration states, and not the other way.

 

May please some of the most experienced contributors in this forum confirm this? and if that's the case, if it were possible that someone from the company could correct such and important error in the documentation?

Best regards.

 

 

1 REPLY 1
Highlighted

Re: Is description of YARN Scheduler Queue Mapping Override property wrong in HDP Docs?

New Contributor

This is not an answer because I have the same doubt after reading and reading official docs

 

Enable Override of Default Queue Mappings (cloudera.com)

 

It could be easier to understand if someone could answer the following questions based on example included in the previous link.

Queue Mappings --> u:maria:engineering

 

 a) If Queue Mappings Override is enable. What does it mean? what are the consequences?

 b) If Queue Mappings Override is disable. What does it mean? what are the consequences?

 

Thanks in advance