Created on 12-21-2016 03:17 AM - edited 08-18-2019 04:18 AM
When I running yarn app, I often get some app in queue( state ACCEPTED). But you can see I still have free resource.
So why app in queue ?
Created 12-21-2016 05:15 AM
@Hoang Le if you look at the Max Application Master Resources,Used Application Master Resources both are same(4G), so this is hitting resource cap for application master and waiting to allocate Application Master container.
Created 12-21-2016 05:15 AM
@Hoang Le if you look at the Max Application Master Resources,Used Application Master Resources both are same(4G), so this is hitting resource cap for application master and waiting to allocate Application Master container.
Created 12-21-2016 05:37 AM
@Rajkumar Singh So how can I define Max Application Master Resources.
In default Configured Max Application Master Limit is set to 20%. If I increase figure, Max Application Master Resources increase. But how this Max Application Master Resources is calculated ?
Created 12-21-2016 05:41 AM
@Hoang Le the maximum is calculated based on yarn.scheduler.capacity.maximum-am-resource-percent, seems you are launching am for one of the job which is configured to get 4GB of memory thats why you are seeing used am resources as 4GB which is the max-am-resource.
Created on 12-21-2016 06:36 AM - edited 08-18-2019 04:18 AM
I find this doc from hortonworks
and this from apache hadoop:
I know if I change yarn.scheduler.capacity.maximum-am-resource-percent then Max Application Master Resources will change. But I want to know which configure it depend to calculate.
For example if true I set yarn.scheduler.capacity.maximum-am-resource-percent = 1 (100%)
then Max Application Master Resources balance with yarn.nodemanager.resource.memory-mb
Created 12-21-2016 06:56 AM
@Hoang Le this is to Get the maximum am resource percent per queue configured capacity. yarn.nodemanager.resource.memory-mb is the memory that a nodemanger announce to RM.