Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

What are the best config memory and vcore in YARN and mapred multi-node cluster?

New Contributor

I have a Hadoop cluster with these specifications:


PC1 --> master: Number of cores: 40 RAM 64G SSD250G
PC2 --> slave: Number of cores: 30 RAM 32G SSD128G
PC3 --> slave2: Number of cores: 30 RAM 32G SSD128G


in each slave, I'll create 2 containers for data nodes, at a total of 5 nodes will setup. Please guide me on what should be the best configuration for memory and vcore on yarn-site.xml and mapred-site.xml for better performance.




Rising Star

As a rule of thumb assign 80% of the node resources to YARN.
Please go through this to modify the config according to your need.