Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎03-17-2015

MapReduce job in CDH 5.1.0 is running locally, not submitted to cluster

Hi,

 

I have recently installed CDH 5.1.0 using Cloudera Manager. When using MR2 jobs for testing services it seems my MR jobs are running locally and jobs are not submitted to cluster. I cannot see the jobs submitted in my RM UI also.

 

When i run Hadoop MR job with my user like hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.3.0-cdh5.1.0.jar grep /user/sreejith/sample.txt /user/sreejith/output 'a-z' it run locally.

 

MR logs in console shows INFO mapred.JobClient: Running job: job_local1263402498_0001

 

My configurations in mapred-site and yarn-site seems to be correct 

Configurations in mapred-site.xml is :

<property>
<name>mapreduce.jobhistory.address</name>
<value>JobHistory:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>JobHistory:19888</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.https.address</name>
<value>JobHistory:19890</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>yarn.app.mapreduce.am.staging-dir</name>
<value>/user</value>
</property>

 

yarn-site.xml:

<property>
<name>yarn.resourcemanager.address</name>
<value>ResourceManager:8032</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>ResourceManager:8033</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>ResourceManager:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>ResourceManager:8031</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>ResourceManager:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.https.address</name>
<value>ResourceManager:8090</value>
</property>

 

Can anyone point me in right direction what is going wrong here ?

 

Thanks,

Sreejith

Announcements