Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

MapReduce job in CDH 5.1.0 is running locally, not submitted to cluster

Highlighted

MapReduce job in CDH 5.1.0 is running locally, not submitted to cluster

New Contributor

Hi,

 

I have recently installed CDH 5.1.0 using Cloudera Manager. When using MR2 jobs for testing services it seems my MR jobs are running locally and jobs are not submitted to cluster. I cannot see the jobs submitted in my RM UI also.

 

When i run Hadoop MR job with my user like hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.3.0-cdh5.1.0.jar grep /user/sreejith/sample.txt /user/sreejith/output 'a-z' it run locally.

 

MR logs in console shows INFO mapred.JobClient: Running job: job_local1263402498_0001

 

My configurations in mapred-site and yarn-site seems to be correct 

Configurations in mapred-site.xml is :

<property>
<name>mapreduce.jobhistory.address</name>
<value>JobHistory:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>JobHistory:19888</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.https.address</name>
<value>JobHistory:19890</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>yarn.app.mapreduce.am.staging-dir</name>
<value>/user</value>
</property>

 

yarn-site.xml:

<property>
<name>yarn.resourcemanager.address</name>
<value>ResourceManager:8032</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>ResourceManager:8033</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>ResourceManager:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>ResourceManager:8031</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>ResourceManager:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.https.address</name>
<value>ResourceManager:8090</value>
</property>

 

Can anyone point me in right direction what is going wrong here ?

 

Thanks,

Sreejith