Hi @Michael Bronson All services logs can be found at the location mentioned in the yarn-env.sh (can be found under Hadoop Conf files, usually /etc/hadoop/conf/) on the respective nodes. Hence please check location mentioned in yarn-env.sh on the master02 machine. Once we check the logs, it will be easier to figure out the exact reason for failure. Example yarn-env.sh export YARN_LOG_DIR=/grid/0/log/yarn/ ... View more

Hi @PJ All services logs can be found at the location mentioned in the yarn-env.sh (can be found under Hadoop Conf files, usually /etc/hadoop/conf/) on the respective nodes. Example yarn-env.sh export YARN_LOG_DIR=/grid/0/log/yarn/ ... View more

Hi @nur majid, In mapred queue -list output, default capacity and maximum capacity are with respect to Cluster' resources whereas current capacity is with respect to Queue's resources. Example - Assume cluster's resources - 10GB Default Queue default capacity is 20% of cluster's resources which is 2GB But default queue can go upto 30% of cluster's resources [maximum capacity] which is 3GB Current capacity is 101% of Queue's capacity [Remember , queue's capacity is 2GB] , so it will come around 2.02GB which is less than 3GB[maximum capacity]. ... View more

Hi @Anurag Mishra, If you want to change yarn-site.xml in the server, then it is also fine. But you need to restart the services on your own instead of using ambari .(if you use ambari to restart services, then as mentioned by Jay, your changes will get wiped off) WARNING : It so happens, that for some properties in yarn-site.xml, it requires restart of more than one service , in which case, you need to copy the change in all service nodes and restart them. ADVANTAGE OF AMABRI - In case of ambari, its just one point change, it will figure out which services require that change and suggest restart of all the services required. ... View more

Hi @Rahul Gupta, Yes, a single node can have more than one application masters when we have several applications running; where each application master belongs to a unique application. None of the application master will be aware of the presence of the other. More info on AM https://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/ ... View more

Hi @Gunjan Dhawas For point 2 - Also as you mentioned its nodemanager which will communicate with container, so can nodemanager directly communicate with containers which are running on different nodes or it will go through RM to get container information. Nodemanagers are basically YARN’s per-node agent, and takes care of the individual compute nodes in a Hadoop cluster. This includes keeping up-to date with the ResourceManager (RM), overseeing containers’ life-cycle management; monitoring resource usage (memory, CPU) of individual containers, tracking node-health, log’s management and auxiliary services which may be exploited by different YARN applications. So Nodemanagers are the nodes on which containers are launched. So yes, nodemanagers directly monitors the containers and their resource consumption. For point 1 - "The application code executing within the container then provides necessary information (progress, status etc.) to its ApplicationMaster via an application-specific protocol.", so how the application master monitor the status of containers which are running on different node than applicatioMaster. Once the applicationMaster negotiates the resources with RM, it will launch the container by providing container launch specification to the NodeManager. The launch specification includes the necessary information to allow the container to communicate with the ApplicationMaster itself. Thus ApplicationMaster gets the progress/status via application-specific protocol provided in the container launch specification ... View more