we have multiple systems with different cpu and ram each
the cpu can be used to utilize 100%
but i could not do it for the ram
You can manually configure your memory allocation settings based on the hardware specifications. Please refer below links for more tuning up RAM in yarn configuration.
i want to configure it via the ambari configuraion manager, so theat each host will use 100% ram and CPU
For changing memory value through ambari configuration manager, login into ambari UI,
Yarn —> Configs —> Settings —> memory —> Memory allocated for all YARN containers on a node and Minimum / Maximum Container Size memory.
and for advanced turning of yarn modify
Yarn —> Configs —> Settings —>Advanced —> Advanced yarn-site
i looking for an option to set 100% and not to set fixd ram or cpu
If you set 100% ram only for yarn then other services will get crash / GC issue. It's not recommendable to set all physical memory to one component.
ok, then configure a 90% mark, the main issue that i want to manage single configuraion file for all hardware
For example your physical memory size is 10GB you need to allocated for every container request at the RM, is around 9GB (90% of your physical memory) then you can mention in below parameter
Similarly amount of physical memory allocated for containers in node manager, that can be mention using below parameter.
These all settings are manage by single configuration file I.e., yarn-site.xml which you can modify via ambari under Yarn —> Configs —> Settings —>Advanced —> Advanced yarn-site
I hope that the above answers your questions. Please accept the answer you found most useful.
i know to to configure it,
for example now i have 3 types of hardares:
3 computers with 12 core cpu and 64gb ram
6 computers with 16 core and 64gb ram
and 4 computers with 48 core and 256gb ram
and i dont want to manage 3 configuraion files , i want to manage one, for instance each computer will allocate all cpus and 90% of ram for yarn tasks
For your question, answer is No. As of now we can't mange memory of individual node level using percentage. Currently yarn support only CPU, using below configuration.
Hi @Ilia K,
You need to create the new config group under Yarn config and add the new nodemanagers into that group.
Then override the yarn nodemanger properties for cpu and memory.
i just wanted to prevent managaing multuiple configurations