Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

job only submit to local not to cluster

job only submit to local not to cluster

New Contributor

I am testing CDH5(HDFS2 + MRV1) with 11 virtual Nodes(1 master(namenode + jobtracker) + 10 worker node(datanode+ tasktracker)) with CentOS5.8

Following the CDH5 installion guide  http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/c...

However hit a weird issue that “all jobs only can submit to localhost not to cluster(jobtracker)” try with two case (client combine with master and worker node).

Even if run job with “–jt jobtracker:8021” still can’t resolve this issue.

Do you hit the same issue?

 

mapred-site.xml:

<property>
   <name>mapred.job.tracker</name>
   <value>namenode:8021</value>
   <description>The host and port that the MapReduce job tracker runsat.</description>
</property>

 

core-site.xml:

<property>
        <name>fs.defaultFS</name>
        <value>hdfs://namenode:8020</value>
        <description>
            The name of the default file system. A URI whose scheme and authority determine the FileSystem implementation.
        </description>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131072</value>
    </property>

 

hdfs-site.xml

   <property>
       <name>dfs.replication</name>
       <value>3</value>
   </property>
   <property>
       <name>dfs.block.size</name>
       <value>67108864</value>
   </property>
   <property>
        <name>dfs.http.address</name>
        <value>namenode:8087</value>
   </property>
   <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:///hadoop/hadoop-data/namenodeyum</value>
   </property>
   <property>
        <name>dfs.webhdfs.enabled</name>
        <value>true</value>
   </property> 

1 REPLY 1

Re: job only submit to local not to cluster

Master Collaborator

I have moved this topic to the MapReduce discussion board.

 

Usually when this happens, it's because you have not set up your environment correctly so that when you submit the MR job, it's not picking up your mapred-site.xml file and therefore selects the local job runner.

 

Maybe this page will help you get the environment variables/conf files set up correctly:

 

http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/SingleCluster.html#Setting_up...