Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Hive Exception = Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapred

avatar
Explorer

I have the following installed on a client application machine

 

[root@cdn-ui tomcat6]# yum list installed | grep hive
hive.noarch             0.13.1+cdh5.2.1+235-1.cdh5.2.1.p0.9.el6
hive-jdbc.noarch        0.13.1+cdh5.2.1+235-1.cdh5.2.1.p0.9.el6
perl-Archive-Extract.x86_64
perl-Archive-Tar.x86_64 1.58-129.el6    @Anaconda-CentOS-201303020151.x86_64/6.4
[root@cdn-ui tomcat6]# yum list installed | grep cdh5
avro-libs.noarch        1.7.6+cdh5.2.0+67-1.cdh5.2.0.p0.41.el6
                                        @Cloudera-cdh5
bigtop-jsvc.x86_64      0.6.0+cdh5.2.0+572-1.cdh5.2.0.p0.33.el6
                                        @Cloudera-cdh5
bigtop-utils.noarch     0.7.0+cdh5.2.0+0-1.cdh5.2.0.p0.40.el6
                                        @Cloudera-cdh5
hadoop.x86_64           2.5.0+cdh5.2.0+551-1.cdh5.2.0.p0.46.el6
                                        @Cloudera-cdh5
                        2.5.0+cdh5.2.0+551-1.cdh5.2.0.p0.46.el6
                                        @Cloudera-cdh5
hadoop-client.x86_64    2.5.0+cdh5.2.0+551-1.cdh5.2.0.p0.46.el6
                                        @Cloudera-cdh5
hadoop-hdfs.x86_64      2.5.0+cdh5.2.0+551-1.cdh5.2.0.p0.46.el6
                                        @Cloudera-cdh5
hadoop-mapreduce.x86_64 2.5.0+cdh5.2.0+551-1.cdh5.2.0.p0.46.el6
                                        @Cloudera-cdh5
hadoop-yarn.x86_64      2.5.0+cdh5.2.0+551-1.cdh5.2.0.p0.46.el6
                                        @Cloudera-cdh5
hive.noarch             0.13.1+cdh5.2.1+235-1.cdh5.2.1.p0.9.el6
                                        @Cloudera-cdh5
hive-jdbc.noarch        0.13.1+cdh5.2.1+235-1.cdh5.2.1.p0.9.el6
                                        @Cloudera-cdh5
parquet.noarch          1.5.0+cdh5.2.0+35-1.cdh5.2.0.p0.34.el6
                                        @Cloudera-cdh5
parquet-format.noarch   2.1.0+cdh5.2.0+4-1.cdh5.2.0.p0.43.el6
                                        @Cloudera-cdh5
sentry.noarch           1.4.0+cdh5.2.1+88-1.cdh5.2.1.p0.10.el6
                                        @Cloudera-cdh5
zookeeper.x86_64        3.4.5+cdh5.2.0+82-1.cdh5.2.0.p0.40.el6
                                        @Cloudera-cdh5

 

when I run the command Hive this is what I am getting

 

[root@cdn-ui tomcat6]# hive
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:190)
        at org.apache.hadoop.hive.shims.ShimLoader.createShim(ShimLoader.java:147)
        at org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:142)
        at org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:101)
        at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:249)
        at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:58)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:649)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        ... 14 more

 

I also updated the conf file on the machine to be identical to the one on the cluster.

 

Do you know what is causing the excpetion ?

1 ACCEPTED SOLUTION

avatar
Explorer

I resolved this by using cloudera manager to do the installation on the machine.

I had to add the machine to the list of hosts then configure a template to include hive gateway and then added and started the role on the machine.

I also uninstalled all my yum installation.

Workes fine now

View solution in original post

1 REPLY 1

avatar
Explorer

I resolved this by using cloudera manager to do the installation on the machine.

I had to add the machine to the list of hosts then configure a template to include hive gateway and then added and started the role on the machine.

I also uninstalled all my yum installation.

Workes fine now