@satya gaurav Please find the details below: 1. How multiple reducers writing the output ? can multiple reducers write the output in a single output file? Or we have to write an intermediate reducer to do so? i just want to know that how we can get a single output file from multiple reducers. ->By default each reducer will generate a separate output file like part-0000 and this output will be stored in HDFS. if we want to merge all the reducers output to single file, then explicitly we have write our own code using MultipleOutputs or using hadoop -fs getmerge command 2. can map and reduce task run in the same container or more than one map task or reduce task can run in the same container?if yes then how? How container is assigning for map and reduce task? ->Yes map and reduce task run in the same container but not parallel. In mapredce V1 we have fixed mappers and reducer slots and we have to run map tasks in mapper slots and reducer tasks in reducer slots only. But in Mapreduce V2 we have option to run either map/reduce tasks in either mapper/reducer container. -> We can't run more than 1 map/reduce tasks at a time in the same container ... View more

@satya gaurav number of reducers is determined exactly by mapreduce.job.reduces. This is not just a recommendation. If you have specified a higher number of reducers, container allocation is still done based on queue size for that application. This is determined by your scheduler. Just because you request more than you should doesn't mean that those resources will be allocated. Your reducers will be waiting in queue until other complete. To get more details, you need to understand schedulers (capacity scheduler to be precise). Minimum container size is given by yarn.scheduler.minimum-allocation-mb (your request for less than this value will still result in a container with this minimum value and not a value you specify if its less than this). Similarly there is an upper limit given by yarn.scheduler.maximum-allocation-mb. Guess what happens if you request more than this? You don't get it. You get assigned this value if you request memory more than this. There are similar settings for core. This is at the cluster level. For each node, containers are allocated by Node Manager which of course is asking Resource Manager to do its job. yarn.nodemanager.resource.memory-mb is how much memory a container will allocate and yarn.nodemanager.resource.cpu-vcores is for CPU ... View more