Created on 11-17-2016 07:29 PM - edited 09-16-2022 03:48 AM
Hi, I'm confused about how to config memory in YARN cluster.
So far, I have some machines each with 64GB physical memory. Because all machines are 64GB, I can set a unified yarn.nodemanager.resource.memory-mb=60GB。
If I want to add new and better machines with 128GB physical memory. How should I set the memory configuration in YARN.
If I set yarn.nodemanager.resource.memory-mb=120GB, will this affect the old machines?
If I set yarn.nodemanager.resource.memory-mb=60GB, will this waste the new machines' resource?
Is there a relative ration to fit each machine adaptively? For example, set yarn.nodemanager.resource.memory-mb=0.8*physical_memory?
Thanks.
Created 11-17-2016 08:23 PM
Created 11-17-2016 08:23 PM
Created 11-17-2016 09:22 PM
Created 11-17-2016 09:56 PM
Created 11-17-2016 09:58 PM