@satya gaurav Number of reducers is determined by mapreduce.job.reduces. Typically set to 99% of the cluster's reduce capacity, so that if a node fails the reduces can still be executed in a single wave. As for Application Master, you first need to understand YARN components. There is resource manager which has two main components: Scheduler and Application Manager (NOT Application Master). The ApplicationsManager is responsible for accepting job-submissions, negotiating the first container for executing the application specific ApplicationMaster and provides the service for restarting the ApplicationMaster container on failure. The per-application ApplicationMaster has the responsibility of negotiating appropriate resource containers from the Scheduler, tracking their status and monitoring for progress. Question: suppose if there is 2gb ram is available in a nodemanager and we submitted a job with 3gb ram then how job will run or it will not run. You need to understand how capacity scheduler works. There are assigned queues for the application(job) and capacity scheduler guarantees certain number of resources available to that application. If resources are available from other queues then your job can borrow those resources. This means if the queue is configured to use 2GB but if your job needs more resources, it can borrow. Please see the section titled "YARN Walkthrough" on following page: http://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/ screen-shot-2017-02-21-at-121739-am.png
... View more
