- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
how to calculate the yarn.nodemanager.resource.cpu-vcores in ambari cluster
- Labels:
-
Apache Ambari
-
Apache Hadoop
Created ‎01-31-2018 02:49 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
as I read from the doc ,yarn.nodemanager.resource.cpu-vcores are by default ~80% of total vCPUs available on the machine
so just to clear that
in case we have ambari cluster with 3 master machine and 3 workers )( machines , and each worker have 8 core
then the calculate will be 3X8=32 ?
or maybe yarn.nodemanager.resource.cpu-vcores should be calculate per worker machine as: 1X8=8 ?
or else?
Created ‎01-31-2018 04:59 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It will be 80% of your vCPUs on the machine. If you have 3 machines with 16 cores , then it will be 80% of 16 ie 12.8. It will take the floor value of 12.8 and set it as 12.
You can get the cpus by running the command
cat /proc/cpuinfo | grep processor | wc -l
If there are multiple node managers with different vCPU count , it will take the max of all the vCPUs
Thanks,
Aditya
Created ‎01-31-2018 04:59 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It will be 80% of your vCPUs on the machine. If you have 3 machines with 16 cores , then it will be 80% of 16 ie 12.8. It will take the floor value of 12.8 and set it as 12.
You can get the cpus by running the command
cat /proc/cpuinfo | grep processor | wc -l
If there are multiple node managers with different vCPU count , it will take the max of all the vCPUs
Thanks,
Aditya
