Support Questions

Find answers, ask questions, and share your expertise

Namenode and Datanode capacity planning


To Hadoop Guru's,

I am new in planning cluster and need some directions in doing some capacity planing for Hadoop Cluster.

Production cluster will be on

1) Node 1: Namenode

2) Node 2: Resouce Manager Node

3) Node 3: Standby Name node

4) Datanodes

But not sure how much RAM will be required for namenode and each datanode, as well as no of CPU's. The block size is 128MB. Let me know what else needs to be consider to select the RAM size and # of CPU's.

Thank you,



Rising Star

@Sachin Ambardekar There is documentation at that discusses overall cluster planning. Things like memory sizing, configurations for different types of nodes (masters vs. workers), and other hardware considerations are detailed at


As per link,

"Depending on the number of cores, your slave nodes typically require 24 GB to 48 GB of RAM for Hadoop applications"

Does this mean that each datanode should have 24GB - 48GB RAM and quad core CPU?

Please advise.

Thank you,

Sachin A


Any update?

Thank you,

Sachin A

Super Mentor

@Sachin Ambardekar

- The NameNode heap size depends on many factors, such as the number of files, the number of blocks, and the load on the system. So you can refer to know how much Heap will be usually needed for the NameNode based on the number of files and same thing will apply on the "StandBy NameNode". So you can plan for Xmx heap memory and RAM on that namenode host accordingly.

- Similarly for YARN (like resource Manager) you can use the HDP utility script is the recommended method for calculating HDP memory configuration settings, also information about manually calculating YARN and MapReduce memory configuration settings is also provided for reference. See below link


#  python -c 16 -m 64 -d 4 -k True

Using cores=16 memory=64GB disks=4 hbase=True
Profile: cores=16 memory=49152MB reserved=16GB usableMem=48GB disks=4 
Num Container=8
Container Ram=6144MB 
Used Ram=48GB
Unused Ram=16GB
mapreduce.reduce.memory.mb=6144 =-Xmx4096m 


@Sachin Ambardekar, The doc above may be slightly dated. Rule of thumb, 4GB per core seems to be the sweet spot for memory intensive workloads which are getting more common nowadays.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.