Created on 01-29-2015 08:50 AM - edited 09-16-2022 02:20 AM
Hello,
I'm trying to join one table that is stored in hbase with another table that is stored as json.
I added the needed jars that I added while creating the hbase table and the serde jar for the join.
When I join the two tables in the hive beeswax everithing works fine.
However, when I do it in an oozie action I get:
15/01/29 17:13:30 INFO mr.ExecDriver: adding libjars: /user/agherman/scripts/General/Hive/json-serde-1.3-jar-with-dependencies.jar java.lang.NoClassDefFoundError: org/cliffc/high_scale_lib/Counter at org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureJobConf(HBaseStorageHandler.java:500) at org.apache.hadoop.hive.ql.plan.PlanUtils.configureJobConf(PlanUtils.java:849) at org.apache.hadoop.hive.ql.plan.MapWork.configureJobConf(MapWork.java:503) at org.apache.hadoop.hive.ql.plan.MapredWork.configureJobConf(MapredWork.java:68) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:368) at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1554) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1321) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1139) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:952) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633) at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323) at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39) at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: java.lang.ClassNotFoundException: org.cliffc.high_scale_lib.Counter at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
Where should I look for?
Thank you!
Created 01-30-2015 02:19 AM
I added the high-scale-lib-1.1.1.jar in order to make it work. 😄
Created 01-30-2015 02:19 AM
I added the high-scale-lib-1.1.1.jar in order to make it work. 😄
Created 02-05-2015 03:15 PM
Thanks for letting us know the solution, Alina!
Created 04-18-2017 11:10 PM
i have an error after that step
hive> insert overwrite table hbase_table_2 select concat(ename,":",eno)as key,ename,eno,eage,esalary from emp;
Query ID = usr_20170419111919_299ad34e-a846-45fa-8334-84291d3dc9f1
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Job running in-process (local Hadoop)
Hadoop job information for Stage-0: number of mappers: 0; number of reducers: 0
2017-04-19 11:20:15,620 Stage-0 map = 0%, reduce = 0%
Ended Job = job_local1415977764_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:8080/
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec