Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

CDH 5.4.4 - Oozie/Scoop Action --as-parquetfile: NoClassDefFound DatasetKeyOutputFormat

avatar
Explorer

When trying to run a Scoop import action with --as-parquetfile, I get a NoClassDefFoundError error. I am using the system shared lib path, and I notice that the kite sdk is installed correctly. the oozie admin lists the jars that should be available on the classpath.

 

I cannot update the shared library through the oozie admin command either (it just returns null)

 

Update Shared Lib

 

[root@quickstart ~]# export OOZIE_URL=http://quickstart.cloudera:11000/oozie
[root@quickstart ~]# oozie admin -sharelibupdate
null
[root@quickstart ~]# echo $?
0

 

 

Oozie Log

 

2015-07-27 15:13:31,585  WARN ParameterVerifier:523 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] The application does not define formal parameters in its XML definition
2015-07-27 15:13:31,686  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@:start:] Start action [0000018-150727074606033-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:31,687  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@:start:] [***0000018-150727074606033-oozie-oozi-W@:start:***]Action status=DONE
2015-07-27 15:13:31,687  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@:start:] [***0000018-150727074606033-oozie-oozi-W@:start:***]Action updated in DB!
2015-07-27 15:13:31,774  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-views] Start action [0000018-150727074606033-oozie-oozi-W@sqoop-import-views] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:31,775  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-views] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-views***]Action status=DONE
2015-07-27 15:13:31,775  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-views] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-views***]Action updated in DB!
2015-07-27 15:13:31,852  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] Start action [0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:35,279  WARN JobSubmitter:153 - SERVER[quickstart.cloudera] Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2015-07-27 15:13:35,285  WARN JobSubmitter:261 - SERVER[quickstart.cloudera] No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2015-07-27 15:13:35,993  INFO SqoopActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] checking action, hadoop job ID [job_1438008296447_0031] status [RUNNING]
2015-07-27 15:13:35,996  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories***]Action status=RUNNING
2015-07-27 15:13:35,996  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories***]Action updated in DB!
2015-07-27 15:13:36,030  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] Start action [0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:40,029  WARN JobSubmitter:153 - SERVER[quickstart.cloudera] Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2015-07-27 15:13:40,033  WARN JobSubmitter:261 - SERVER[quickstart.cloudera] No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2015-07-27 15:13:40,904  INFO SqoopActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] checking action, hadoop job ID [job_1438008296447_0032] status [RUNNING]
2015-07-27 15:13:40,911  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products***]Action status=RUNNING
2015-07-27 15:13:40,911  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products***]Action updated in DB!
2015-07-27 15:13:57,716  INFO CallbackServlet:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] callback for action [0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories]
2015-07-27 15:13:58,048  INFO SqoopActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] action completed, external ID [job_1438008296447_0031]
2015-07-27 15:13:58,056  WARN SqoopActionExecutor:523 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
2015-07-27 15:13:58,057  WARN SqoopActionExecutor:523 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] Launcher exception: org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
java.lang.NoClassDefFoundError: org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
	at org.apache.sqoop.mapreduce.DataDrivenImportJob.getOutputFormatClass(DataDrivenImportJob.java:187)
	at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:94)
	at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259)
	at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:668)
	at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
	at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:196)
	at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:176)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
	at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.kitesdk.data.mapreduce.DatasetKeyOutputFormat
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	... 34 more

 

Job.properties

 

# Workflow Parameters
oozie.use.system.libpath=true

 

Output of -shareliblist sqoop

 

[cloudera@quickstart ~]$ oozie admin -shareliblist sqoop
[Available ShareLib]
sqoop
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/ST4-4.0.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/activation-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/ant-1.8.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/ant-launcher-1.8.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/antlr-2.7.7.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/antlr-runtime-3.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/apache-curator-2.6.0.pom
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/apache-log4j-extras-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/asm-3.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/asm-commons-3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/asm-tree-3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/avro-ipc-tests.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/avro-ipc.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/avro-mapred-hadoop2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/bonecp-0.7.1.RELEASE.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/calcite-avatica-1.0.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/calcite-core-1.0.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/calcite-linq4j-1.0.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-compiler-2.7.6.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-compress-1.4.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-dbcp-1.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-io-2.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-jexl-2.1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-pool-1.5.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/curator-client-2.7.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/curator-framework-2.6.0.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/datanucleus-api-jdo-3.2.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/datanucleus-core-3.2.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/datanucleus-rdbms-3.2.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/eigenbase-properties-1.1.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/findbugs-annotations-1.3.9-1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/geronimo-annotation_1.0_spec-1.1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/geronimo-jaspic_1.0_spec-1.0.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/geronimo-jta_1.1_spec-1.1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/groovy-all-2.1.6.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/guava-11.0.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hbase-annotations.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hbase-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-ant.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-cli.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-exec.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-hcatalog-core.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-metastore.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-serde.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-service.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims-0.23.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims-scheduler.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hsqldb-1.8.0.10.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/htrace-core-3.1.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/httpclient-4.2.5.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/httpcore-4.2.5.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jackson-annotations-2.3.0.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jackson-core-2.3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jackson-databind-2.3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/janino-2.7.6.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jdo-api-3.0.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jersey-servlet-1.14.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jetty-all-7.6.0.v20120127.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jline-2.11.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jpam-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jsr305-1.3.9.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jta-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/kite-data-core.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/kite-hadoop-compatibility.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/libfb303-0.9.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/libthrift-0.9.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/logredactor-1.0.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/mail-1.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/netty-3.4.0.Final.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/oozie-sharelib-sqoop-4.1.0-cdh5.4.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/opencsv-2.3.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-avro.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-column.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-encoding.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-format.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-generator.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-hadoop-bundle.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-hadoop.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-jackson.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/snappy-java-1.0.4.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/sqoop.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/stringtemplate-3.2.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/xz-1.0.jar 

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor
It looks like "kite-data-mapreduce.jar" is missing from the Oozie sharelib. We're tracking this internally. Until then, could you upload this to the sharelib or add it to the Sqoop action as a "file"?

View solution in original post

3 REPLIES 3

avatar
Expert Contributor
It looks like "kite-data-mapreduce.jar" is missing from the Oozie sharelib. We're tracking this internally. Until then, could you upload this to the sharelib or add it to the Sqoop action as a "file"?

avatar
Explorer
Which version of this library should be used? sharelib doesn't have version numbers for these jars.

avatar
Expert Contributor
What ever is available in your CDH distribution.

If you're using parcels, take a look at /opt/cloudera/parcles/CDH/jars.

If you're using packages, take a look at /usr/lib/kite.

You can upload that jar to any directory in HDFS and make it available to your workflow by including it using the "file" element (https://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a3.2.2.1_Adding_Files_and_Archives_f... That would probably be the easiest way to test.