#Hive #Hbase java.lang.ClassNotFoundException: org.cliffc.high_scale_lib.Counter

[#1]

Hello,

I'm trying to join one table that is stored in hbase with another table that is stored as json.

I added the needed jars that I added while creating the hbase table and the serde jar for the join.

When I join the two tables in the hive beeswax everithing works fine. 

However, when I do it in an oozie action I get:

15/01/29 17:13:30 INFO mr.ExecDriver: adding libjars: /user/agherman/scripts/General/Hive/json-serde-1.3-jar-with-dependencies.jar
  java.lang.NoClassDefFoundError: org/cliffc/high_scale_lib/Counter
  at org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureJobConf(HBaseStorageHandler.java:500)
  at org.apache.hadoop.hive.ql.plan.PlanUtils.configureJobConf(PlanUtils.java:849)
  at org.apache.hadoop.hive.ql.plan.MapWork.configureJobConf(MapWork.java:503)
  at org.apache.hadoop.hive.ql.plan.MapredWork.configureJobConf(MapredWork.java:68)
  at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:368)
  at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
  at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1554)
  at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1321)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1139)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:952)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
  at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
  at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
  at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
  at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
  at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  Caused by: java.lang.ClassNotFoundException: org.cliffc.high_scale_lib.Counter
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

 Where should I look for?

Thank you!

[#2 · Accepted solution]

I added the high-scale-lib-1.1.1.jar in order to make it work. 😄

[#3]

Thanks for letting us know the solution, Alina!

[#4]

i have an error after that step

hive> insert overwrite table hbase_table_2 select concat(ename,":",eno)as key,ename,eno,eage,esalary from emp;
Query ID = usr_20170419111919_299ad34e-a846-45fa-8334-84291d3dc9f1
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Job running in-process (local Hadoop)
Hadoop job information for Stage-0: number of mappers: 0; number of reducers: 0
2017-04-19 11:20:15,620 Stage-0 map = 0%,  reduce = 0%
Ended Job = job_local1415977764_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:8080/
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-0:  HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec