Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

java.lang.NoClassDefFoundError using Hbase Storage Handler on Hive

Solved Go to solution

java.lang.NoClassDefFoundError using Hbase Storage Handler on Hive

New Contributor

Hi,

 

i want to create an external hive tebale mapped on hbase in my oozie workflow. on cdh4 it works without problems but using the script on cdh5 i got an "java.lang.NoClassDefFoundError" exception. I have added all jars like in the documentation:

 

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/c...

 

I also added the jars with the "hive.aux.jars.path" property at the hive-site.xml

 

Hive script:

 

CREATE EXTERNAL TABLE IF NOT EXISTS hive_table(
 hash BIGINT,
 f1 STRING
 )STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES (
 'hbase.columns.mapping' =':key,cf1:f1')TBLPROPERTIES('hbase.table.name' = 'hbase_table');

 

Error:

 

3029 [main] ERROR hive.ql.exec.DDLTask  - java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/mapreduce/TableInputFormatBase
  at java.lang.ClassLoader.defineClass1(Native Method)
  at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
  at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
  at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
  at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:270)
  at org.apache.hadoop.hive.ql.metadata.Table.getInputFormatClass(Table.java:305)
  at org.apache.hadoop.hive.ql.metadata.Table.<init>(Table.java:98)
  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:991)
  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:894)
  at org.apache.hadoop.hive.ql.exec.DDLTask.dropTable(DDLTask.java:3308)
  at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:278)
  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151)
  at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1485)
  at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1263)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1091)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:357)
  at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:455)
  at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:465)
  at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:746)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
  at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:315)
  at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:276)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38)
  at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapreduce.TableInputFormatBase
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  ... 60 more

 


My Cluster is configured with CDH5.0.2 and CDM5 with Yarn MRv2

 

 

I hope you can help me which jar file is missing

1 ACCEPTED SOLUTION

Accepted Solutions

Re: java.lang.NoClassDefFoundError using Hbase Storage Handler on Hive

New Contributor

I was able to fix this issue by copying the following jars located at /opt/cloudera/parcels/CDH/lib/hive/lib to  /user/oozie/share/lib/lib_{ts}/hive/

 

hbase-client.jar
hbase-common.jar
hbase-hadoop2-compat.jar
hbase-hadoop-compat.jar
hbase-protocol.jar
hbase-server.jar
hive-hbase-handler-0.12.0-cdh5.1.3.jar
hive-hbase-handler.jar

htrace-core.jar

 

I'm using Hadoop 2.3.0-cdh5.1.3 and CDH manager 5.

- Saimon

3 REPLIES 3
Highlighted

Re: java.lang.NoClassDefFoundError using Hbase Storage Handler on Hive

New Contributor

Has anyone an idea what library is missing?

 

It would be great if someone could help me briefly?

thanks a lot :)

 

BG,

Re: java.lang.NoClassDefFoundError using Hbase Storage Handler on Hive

New Contributor

I'm having the same problem. Did you find a fix?

Re: java.lang.NoClassDefFoundError using Hbase Storage Handler on Hive

New Contributor

I was able to fix this issue by copying the following jars located at /opt/cloudera/parcels/CDH/lib/hive/lib to  /user/oozie/share/lib/lib_{ts}/hive/

 

hbase-client.jar
hbase-common.jar
hbase-hadoop2-compat.jar
hbase-hadoop-compat.jar
hbase-protocol.jar
hbase-server.jar
hive-hbase-handler-0.12.0-cdh5.1.3.jar
hive-hbase-handler.jar

htrace-core.jar

 

I'm using Hadoop 2.3.0-cdh5.1.3 and CDH manager 5.

- Saimon

Don't have an account?
Coming from Hortonworks? Activate your account here