I have set "yarn.nodemanager.delete.debug-delay-sec" to 6000,and the container log dir is :   <property> <name>yarn.nodemanager.log-dirs</name> <value>/hadoop/hadoop-2.0.0-cdh4.5.0/yarn/containers</value> </property> <property> <description>Where to aggregate logs</description> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/var/log/hadoop-yarn/app</value> </property>   The dir /hadoop/hadoop-2.0.0-cdh4.5.0/yarn/containers has nothing after  running  the task,I found the configruation in yarn-site.xml never take effect. ... View more

I found  the resourcemanager never read configuration from yarn-site.xml  and mapred-site.xml,here is  my configturation: mapred-site.xml: <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>bigdata1:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>bigdata1:19888</value> </property> <property> <name>mapreduce.shuffle.port</name> <value>8070</value> </property> </configuration>     yarn-site.xml:   <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>bigdata1:8031</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>bigdata1:8032</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>bigdata1:8030</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>bigdata1:8033</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>bigdata1:8088</value> </property> <property> <description>Classpath for typical applications.</description> <name>yarn.application.classpath</name> <value>$HADOOP_CONF_DIR,$YARN_HOME/*,$YARN_HOME/lib/*,$YARN_HOME/share/hadoop/common/*,$YARN_HOME/share/hadoop/yarn,$YARN_HOME/share/hadoop/yarn/lib/*,$YARN_HOME/share/hadoop/mapreduce/*,$YARN_HOME/share/hadoop/mapreduce/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,$YARN_HOME/share/hadoop/common/lib/*</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce.shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.dispatcher.exit-on-error</name> <value>true</value> </property> <property> <name>yarn.nodemanager.local-dirs</name> <value>/hadoop/hadoop-2.0.0-cdh4.5.0/yarn/${user.name}/nm-local-dir</value> </property> <property> <name>yarn.nodemanager.log-dirs</name> <value>/hadoop/hadoop-2.0.0-cdh4.5.0/yarn/containers</value> </property> <property> <description>Where to aggregate logs</description> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/var/log/hadoop-yarn/app</value> </property> <property> <name>yarn.app.mapreduce.am.staging-dir</name> <value>/user</value> </property>   </configuration>     ... View more

Yes,it exists,and the privilege is 777.The following is from another example,it is the same as the previous one:   [bigdata1 mapreduce]#hadoop jar ./hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar pi 5 10 Number of Maps = 5 Samples per Map = 10 Wrote input for Map #0 Wrote input for Map #1 Wrote input for Map #2 Wrote input for Map #3 Wrote input for Map #4 Starting Job 14/03/19 21:37:54 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. 14/03/19 21:37:54 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. 14/03/19 21:37:54 INFO input.FileInputFormat: Total input paths to process : 5 14/03/19 21:37:54 INFO mapreduce.JobSubmitter: number of splits:5 14/03/19 21:37:54 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar 14/03/19 21:37:54 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 14/03/19 21:37:54 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 14/03/19 21:37:54 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 14/03/19 21:37:54 WARN conf.Configuration: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 14/03/19 21:37:54 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 14/03/19 21:37:54 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name 14/03/19 21:37:54 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class 14/03/19 21:37:54 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class 14/03/19 21:37:54 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 14/03/19 21:37:54 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/03/19 21:37:54 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class 14/03/19 21:37:54 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 14/03/19 21:37:54 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 14/03/19 21:37:54 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 14/03/19 21:37:55 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1395234357046_0007 14/03/19 21:37:55 INFO client.YarnClientImpl: Submitted application application_1395234357046_0007 to ResourceManager at bigdata1/192.168.65.66:8032 14/03/19 21:37:55 INFO mapreduce.Job: The url to track the job: http://bigdata1:8088/proxy/application_1395234357046_0007/ 14/03/19 21:37:55 INFO mapreduce.Job: Running job: job_1395234357046_0007 14/03/19 21:38:09 INFO mapreduce.Job: Job job_1395234357046_0007 running in uber mode : false 14/03/19 21:38:09 INFO mapreduce.Job: map 0% reduce 0% 14/03/19 21:38:09 INFO mapreduce.Job: Job job_1395234357046_0007 failed with state FAILED due to: Application application_1395234357046_0007 failed 1 times due to AM Container for appattempt_1395234357046_0007_000001 exited with exitCode: 1 due to: .Failing this attempt.. Failing the application. 14/03/19 21:38:09 INFO mapreduce.Job: Counters: 0 Job Finished in 15.584 seconds java.io.FileNotFoundException: File does not exist: hdfs://bigdata1:8020/user/root/QuasiMonteCarlo_TMP_3_141592654/out/reduce-out at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:824) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1704) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1728) at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:314) at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:351) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:360) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) ... View more

When I run the example as the document says,there is something wrong,I don't known what happens,someone give me an idea? My resourcemanager and the nodemangers all work well.   [bigdata1 /hadoop/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce]#hadoop jar ./hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar grep /input /output 'dfs[a-z.]+' 14/03/10 19:02:34 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. 14/03/10 19:02:34 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. 14/03/10 19:02:34 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job or Job#setJar(String). 14/03/10 19:02:34 INFO input.FileInputFormat: Total input paths to process : 7 14/03/10 19:02:35 INFO mapreduce.JobSubmitter: number of splits:7 14/03/10 19:02:35 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 14/03/10 19:02:35 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class 14/03/10 19:02:35 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 14/03/10 19:02:35 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class 14/03/10 19:02:35 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 14/03/10 19:02:35 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 14/03/10 19:02:35 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 14/03/10 19:02:35 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1394435601283_0011 14/03/10 19:02:35 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources. 14/03/10 19:02:35 INFO client.YarnClientImpl: Submitted application application_1394435601283_0011 to ResourceManager at bigdata1/192.168.65.66:8032 14/03/10 19:02:35 INFO mapreduce.Job: The url to track the job: http://bigdata1:8088/proxy/application_1394435601283_0011/ 14/03/10 19:02:35 INFO mapreduce.Job: Running job: job_1394435601283_0011 14/03/10 19:02:41 INFO mapreduce.Job: Job job_1394435601283_0011 running in uber mode : false 14/03/10 19:02:41 INFO mapreduce.Job: map 0% reduce 0% 14/03/10 19:02:41 INFO mapreduce.Job: Job job_1394435601283_0011 failed with state FAILED due to: Application application_1394435601283_0011 failed 1 times due to AM Container for appattempt_1394435601283_0011_000001 exited with exitCode: 1 due to: .Failing this attempt.. Failing the application. 14/03/10 19:02:41 INFO mapreduce.Job: Counters: 0 14/03/10 19:02:41 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. 14/03/10 19:02:41 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. 14/03/10 19:02:42 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job or Job#setJar(String). 14/03/10 19:02:42 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/root/.staging/job_1394435601283_0012 14/03/10 19:02:42 ERROR security.UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE) cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://bigdata1:8020/user/root/grep-temp-1811237380 org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://bigdata1:8020/user/root/grep-temp-1811237380 at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:243) at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:468) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:485) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:369) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1269) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1266) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1266) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1287) at org.apache.hadoop.examples.Grep.run(Grep.java:92) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.examples.Grep.main(Grep.java:101) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) [bigdata1 /hadoop/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce]# ... View more