- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
What are the best config memory and vcore in YARN and mapred multi-node cluster?
- Labels:
-
Apache Hadoop
-
Apache Sqoop
-
Apache YARN
Created ‎07-27-2022 02:26 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a Hadoop cluster with these specifications:
PC1 --> master: Number of cores: 40 RAM 64G SSD250G
PC2 --> slave: Number of cores: 30 RAM 32G SSD128G
PC3 --> slave2: Number of cores: 30 RAM 32G SSD128G
in each slave, I'll create 2 containers for data nodes, at a total of 5 nodes will setup. Please guide me on what should be the best configuration for memory and vcore on yarn-site.xml and mapred-site.xml for better performance.
thanks
Created ‎04-21-2024 10:02 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As a rule of thumb assign 80% of the node resources to YARN.
Please go through this to modify the config according to your need.
