Reply
New Contributor
Posts: 1
Registered: ‎10-19-2016

Getting “NoSuchMethodError: org.apache.hadoop.mapreduce.Job.setJar” error when run mapreduce

I am a cloudera manager newbie, I use cloudera manager v5.8 install CDH clusters,I installed in the cluster hdfs, hbase, hive, MR2 (yarn) and other components, used to run some programs,But when I run a program associated with mapreduce, i get a "NoSuchMethodError: Org.apache.hadoop.mapreduce.Job.setJar " error, I suspect that after the installation of CDH some environment variables are not set so that local mapreduce.client incorrect version, I tried a variety of means to set environment variables can not solve this problem, I also checked cloudera manager mapreduce related configuration parameters, But still can not find the relevant settings to solve this problem.

 

error info:

 

java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.Job.setJar(Ljava/lang/String;)V
at org.apache.kylin.engine.mr.common.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:163)
at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:88)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:88)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Environments:

    root@yinpiao2:~/apache-kylin-1.5.4-cdh5.7-bin/bin# cat /etc/profile
...
    export JAVA_HOME=/usr/java/jdk1.7.0_80
    export PATH=$PATH:$JAVA_HOME/bin
    export MAVEN_HOME=/data/maven
    export HADOOP_CLASSPATH=.:$HADOOP_CLASSPATH:/opt/cloudera-manager/cm-5.8.0/share/cmf/common_jars/hadoop-core-2.6.0-mr1-cdh5.5.0.jar
    export HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
    export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
    export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
    export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
    export ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/zookeeper
    export TOMCAT_HOME=/data/tomcat
    export CATALINA_HOME=/data/tomcat
    export CLASSPATH=.:$CLASSPATH:/usr/share/java/mysql-connector-java-5.1.28.jar:$JAVA_HOME/lib/tools.jar
    export KYLIN_HOME=/home/yinpiao/apache-kylin-1.5.4-cdh5.7-bin
    export hive_dependency=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/conf:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/lib/*:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/jars/hive-hcatalog-core-1.1.0-cdh5.8.0.jar
    export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$KYLIN_HOME/bin:$PATH:$PATH:$TOMCAT_HOME/bin

 

yinpiao@cloudtest45:~$ hadoop version
Hadoop 2.6.0-cdh5.8.0
Subversion http://github.com/cloudera/hadoop -r 042da8b868a212c843bcbf3594519dd26e816e79
Compiled by jenkins on 2016-07-12T22:55Z
Compiled with protoc 2.5.0
From source with checksum 2b6c319ecc19f118d6e1c823175717b5
This command was run using /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/jars/hadoop-common-2.6.0-cdh5.8.0.jar

 

hope someone can answer this question? thank.