Created on 11-04-2014 04:25 AM - edited 09-16-2022 02:11 AM
Hi, I'm trying to switch to CDH5 🙂
I have several nodes each as 32GB ram. Here is my confusion:
1. How to control tasks concurrency? Really, I don't any pools/schedulers right now. I need one huge pool. I don't need any complicity right now, since there is no reason for it. How can I force run all my MR jobs in one huge pool?
2. How to contol memory allocation? I've used mapred.child.java.opts in MR1, in MR2 it doesn't work.
3. I see 100500 setting related to *.memory.mb *.memory.max. I've read description and don't see how they fluence on my MR jobs.
4. Resource manager shows 9 node managers (NM). Each NM has 8Vcores (it's ok, there is 8 HT cores on each node) and 8GB RAM
Why 8GB RAM? I have 32GB per node? How can I change it?
Created 11-11-2014 07:05 AM
this link shoud clear your confusion
https://www.linkedin.com/pulse/article/20140706112523-176301000-yarn-resource-allocation
Created 11-11-2014 07:05 AM
this link shoud clear your confusion
https://www.linkedin.com/pulse/article/20140706112523-176301000-yarn-resource-allocation
Created 11-30-2014 04:40 AM
Created 02-15-2015 05:54 AM