Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NodeManager memory setting best practice?

Solved Go to solution
Highlighted

NodeManager memory setting best practice?

I asked a question regarding OOM in NodeManager log and it turned out I need to increase the NM's heap size.

So that I was wondering if there is any document which advises me how much memory I should assign?

I'm hoping something similar to HDFS NameNode heap size ( http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-... )

Or as it would be hard to estimate, I should just try twice higher and see how it goes?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NodeManager memory setting best practice?

New Contributor

Hajime, the above scripts are for the yarn container and mapreduce memory settings. If you are trying to configure the memory of the nodemanager process itself then that shouldn't need more than 2GB - 4GB. If you are seeing outOfMemory there I suggest you turn on verbose GC for the nodemanager process and review the GC logs.

View solution in original post

5 REPLIES 5
Highlighted

Re: NodeManager memory setting best practice?

New Contributor

Hi Hajime, typically we set the NodeManager heap to 2GB - 4GB - I haven't had to set it higher than that. What is it currently set to?

-Koelli

Highlighted

Re: NodeManager memory setting best practice?

2GB (i wonder why this system has min char limit which doesn't allow me to just type "2GB")

Highlighted

Re: NodeManager memory setting best practice?

@Hajime - The best way to find the nodemanager heap size and other memory settings is to calculate it specifically for your cluster size and hardware spec. Here is the utility that you can use -

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-...

Snippet 

hdp-configuration-utils.sh options where options are as follows:

Table 1.1. hdp-configuration-utils.sh Options

Option

Description

-c CORESThe number of cores on each host.-m MEMORYThe amount of memory on each host in GB.-d DISKSThe number of disks on each host.-k HBASE"True" if HBase is installed, "False" if not.

The output recommendation is in this format -

Using cores=16 memory=64GB disks=4 hbase=True
Profile: cores=16 memory=49152MB reserved=16GB usableMem=48GB disks=4 
Num Container=8
Container Ram=6144MB 
Used Ram=48GB
Unused Ram=16GB
yarn.scheduler.minimum-allocation-mb=6144 
yarn.scheduler.maximum-allocation-mb=49152 
yarn.nodemanager.resource.memory-mb=49152 
mapreduce.map.memory.mb=6144 
mapreduce.map.java.opts=-Xmx4096m 
mapreduce.reduce.memory.mb=6144 
mapreduce.reduce.java.opts=-Xmx4096m 
yarn.app.mapreduce.am.resource.mb=6144 
yarn.app.mapreduce.am.command-opts=-Xmx4096m 
mapreduce.task.io.sort.mb=1792 
tez.am.resource.memory.mb=6144 
tez.am.launch.cmd-opts =-Xmx4096m 
hive.tez.container.size=6144 
hive.tez.java.opts=-Xmx4096m
hive.auto.convert.join.noconditionaltask.size=1342177000
Highlighted

Re: NodeManager memory setting best practice?

Thanks! but if i understand this correctly, it does not show NodeManager itself's heap (Xmx), does it?

Highlighted

Re: NodeManager memory setting best practice?

New Contributor

Hajime, the above scripts are for the yarn container and mapreduce memory settings. If you are trying to configure the memory of the nodemanager process itself then that shouldn't need more than 2GB - 4GB. If you are seeing outOfMemory there I suggest you turn on verbose GC for the nodemanager process and review the GC logs.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here