I can't find the log files from my MapReduce jobs. I'm using MR2 in HortonWorks 2.4.3 sandbox I got from here. In an effort to try to create the logs in one directory, I have set the following environment variables export HADOOP_MAPRED_HOME=/home/hadoop export HADOOP_YARN_HOME=/hadoop/yarn export YARN_LOG_DIR=/hadoop/yarn/log export HADOOP_LOG_DIR=/hadoop/yarn/log export HADOOP_MAPRED_LOG_DIR=/hadoop/yarn/log I'm not sure if setting these in my environment session has any effect when running the job. Presumably I have to set this in Ambari? I do see the job history logs in /var/log/hadoop-mapreduce/mapred/. But I don't see the logs from my map reduce program itself. Following the link in Ambari for MapReduce JobHistory UI takes me to http://<>:19888/jobhistory which shows no jobs. I have tried starting my mapreduce job using yarn jar ./lib/<my>.jar <mapreduce driver class name> <input file name> <output hdfs dir name> <properties file> and hadoop jar ./lib/<my>.jar <mapreduce driver class name> <input file name> <output hdfs dir name> <properties file> Same result with both. Except with the second I get a warning message WARNING: Use "yarn jar" to launch YARN applications. I see nothing in the resource manager (http://<>:8088) or node manager (http://<>:8042) UI According to Simplifying user-logs management and access in YARN, I should be using Application Ids with the yarn commands. But where are these Application Ids set when I invoke the Map Reduce program as I do above? Secondly how am I setting properties like yarn.nodemanager.log-dirs By default it's ${yarn.log.dir}/userlogs Where is ${yarn.log.dir} set? I think this is what I'm missing (along with how to set/get an application Id from a MapReduce program). I think I'm missing something obvious so a nudge in the right direction would be appreciated. ... View more