HI   I have cloudera VM cdh4.4.0.   I am able to submit jobs with the yarn command but not with as a Java application.   The issue i faced is by default the yarn.application.classpath is set as   $HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/share/hadoop/common/*, $HADOOP_COMMON_HOME/share/hadoop/common/lib/*, $HADOOP_HDFS_HOME/share/hadoop/hdfs/*, $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*, $HADOOP_YARN_HOME/share/hadoop/yarn/*, $HADOOP_YARN_HOME/share/hadoop/yarn/lib/*   but with this classpath , after i submitted my job the container fails to start without any message e.g. classnotfounderror, it just silently fails and remove all the container logs with it.   By setting the yarn.application.classpath from yarn-site.xml from hadoop-yarn configuration it works.   Can you tell me what is the best way to set this classpath and if cloudera installation should create links so that the default classpath works?   Sethi   ... View more

 Hi,    I am using cloudera 4.4.0 VM. I am planing to move from mapreduce 0.20 to Yarn.I have done the following.   1. Stopped Mapreduce service 2. start yarn service 3. sudo alternatives --set hadoop-conf  /etc/hadoop/conf.cloudera.yarn1   With this when i run the example wordcount job   yarn jar  hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar wordcount /user/cloudera/input/core-site.xml /tmp/out1   The job is submitted but the mappers are always in pending step and never move forward.There are no error in the application manager log.   The output is copied here   [cloudera@localhost hadoop-mapreduce]$ yarn jar hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar wordcount hdfs://localhost:8020/user/cloudera/input/core-site.xml /tmp/out1 2014-03-18 03:17:36,619 INFO [main] service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. 2014-03-18 03:17:37,157 INFO [main] service.AbstractService(AbstractService.java:start(94)) -Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. 2014-03-18 03:17:41,504 INFO [main] input.FileInputFormat(FileInputFormat.java:listStatus(245)) - Total input paths to process : 1 2014-03-18 03:17:42,118 INFO [main] mapreduce.JobSubmitter(JobSubmitter.java:submitJobInternal(368)) - number of splits:1 2014-03-18 03:17:42,286 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.jar is deprecated. Instead, use mapreduce.job.jar 2014-03-18 03:17:42,304 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 2014-03-18 03:17:42,316 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class 2014-03-18 03:17:42,322 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 2014-03-18 03:17:42,327 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.job.name is deprecated. Instead, use mapreduce.job.name 2014-03-18 03:17:42,329 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class 2014-03-18 03:17:42,334 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 2014-03-18 03:17:42,334 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2014-03-18 03:17:42,339 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2014-03-18 03:17:42,346 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 2014-03-18 03:17:42,352 WARN [main] conf.Configuration(Configuration.java:warnOnceIfDeprecated(824)) - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 2014-03-18 03:17:45,166 INFO [main] mapreduce.JobSubmitter(JobSubmitter.java:printTokens(438)) - Submitting tokens for job: job_1395137802361_0001 2014-03-18 03:17:50,030 INFO [main] client.YarnClientImpl(YarnClientImpl.java:submitApplication(124)) - Submitted application application_1395137802361_0001 to ResourceManager at localhost.localdomain/127.0.0.1:8032 2014-03-18 03:17:51,237 INFO [main] mapreduce.Job (Job.java:submit(1273)) -The url to track the job: http://localhost.localdomain:8088/proxy/application_1395137802361_0001/ 2014-03-18 03:17:51,259 INFO [main] mapreduce.Job(Job.java:monitorAndPrintJob(1318)) - Running job: job_1395137802361_0001 2014-03-18 03:18:19,971 INFO [main] mapreduce.Job(Job.java:monitorAndPrintJob(1339)) - Job job_1395137802361_0001 running in uber mode : false 2014-03-18 03:18:19,999 INFO [main] mapreduce.Job(Job.java:monitorAndPrintJob(1346)) - map 0% reduce 0%   This is where it gets stuck.           ... View more