Created 07-03-2017 07:36 AM
Hi,
H have a WF with two actions, one is a sqoop action and the other is a shell action. I have tested both actions in shell earlier.
now that I need oozie to execute them, it fails in sqoop action with following error:
ive/conf/HiveConf$ConfVars java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf$ConfVars at org.apache.hive.hcatalog.common.HCatConstants.<clinit>(HCatConstants.java:74) at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:299) at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:848) at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:102) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:263) at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:748) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:509) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.main(Sqoop.java:243) at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:202) at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:182) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:51) at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:48) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:242) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf$ConfVars at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 31 more
Sqoop action is meant to import some data from mySQL to a hive partitioned table.
Does anyone know how to sole it?
P.S. I have all my services on a 5 node ambari cluster. my hdp version is 2.5.
Created 07-04-2017 02:25 PM
I solved this issue by adding one line to my job.properties:
oozie.action.sharelib.for.sqoop=sqoop,hive
and two lines to workflow after query in sqoop part:
<file>/user/root/test/shell/hive-site.xml#hive-site.xml</file> <file>/user/root/test/shell/hive-site.xml#tez-site.xml</file>
Created 12-23-2018 05:08 PM
please explain about hive-site.xml
where is hive-site.xml? and you copied this file to "/user/root/test/shell/" ?
pls send an email to me " ahajitrorb@gmail.com "