Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

CDH 5.4.4 - Oozie/Scoop Action --as-parquetfile: NoClassDefFound DatasetKeyOutputFormat

Labels:
avatar
author-rank Justen
Explorer

Created on ‎07-27-2015 03:17 PM - edited ‎09-16-2022 02:35 AM

When trying to run a Scoop import action with --as-parquetfile, I get a NoClassDefFoundError error. I am using the system shared lib path, and I notice that the kite sdk is installed correctly. the oozie admin lists the jars that should be available on the classpath.

 

I cannot update the shared library through the oozie admin command either (it just returns null)

 

Update Shared Lib

 

[root@quickstart ~]# export OOZIE_URL=http://quickstart.cloudera:11000/oozie
[root@quickstart ~]# oozie admin -sharelibupdate
null
[root@quickstart ~]# echo $?
0

 

 

Oozie Log

 

2015-07-27 15:13:31,585  WARN ParameterVerifier:523 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] The application does not define formal parameters in its XML definition
2015-07-27 15:13:31,686  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@:start:] Start action [0000018-150727074606033-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:31,687  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@:start:] [***0000018-150727074606033-oozie-oozi-W@:start:***]Action status=DONE
2015-07-27 15:13:31,687  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@:start:] [***0000018-150727074606033-oozie-oozi-W@:start:***]Action updated in DB!
2015-07-27 15:13:31,774  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-views] Start action [0000018-150727074606033-oozie-oozi-W@sqoop-import-views] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:31,775  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-views] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-views***]Action status=DONE
2015-07-27 15:13:31,775  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-views] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-views***]Action updated in DB!
2015-07-27 15:13:31,852  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] Start action [0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:35,279  WARN JobSubmitter:153 - SERVER[quickstart.cloudera] Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2015-07-27 15:13:35,285  WARN JobSubmitter:261 - SERVER[quickstart.cloudera] No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2015-07-27 15:13:35,993  INFO SqoopActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] checking action, hadoop job ID [job_1438008296447_0031] status [RUNNING]
2015-07-27 15:13:35,996  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories***]Action status=RUNNING
2015-07-27 15:13:35,996  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories***]Action updated in DB!
2015-07-27 15:13:36,030  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] Start action [0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-07-27 15:13:40,029  WARN JobSubmitter:153 - SERVER[quickstart.cloudera] Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2015-07-27 15:13:40,033  WARN JobSubmitter:261 - SERVER[quickstart.cloudera] No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2015-07-27 15:13:40,904  INFO SqoopActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] checking action, hadoop job ID [job_1438008296447_0032] status [RUNNING]
2015-07-27 15:13:40,911  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products***]Action status=RUNNING
2015-07-27 15:13:40,911  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products] [***0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_products***]Action updated in DB!
2015-07-27 15:13:57,716  INFO CallbackServlet:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] callback for action [0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories]
2015-07-27 15:13:58,048  INFO SqoopActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] action completed, external ID [job_1438008296447_0031]
2015-07-27 15:13:58,056  WARN SqoopActionExecutor:523 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
2015-07-27 15:13:58,057  WARN SqoopActionExecutor:523 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000018-150727074606033-oozie-oozi-W] ACTION[0000018-150727074606033-oozie-oozi-W@sqoop-import-view_et_product_categories] Launcher exception: org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
java.lang.NoClassDefFoundError: org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
	at org.apache.sqoop.mapreduce.DataDrivenImportJob.getOutputFormatClass(DataDrivenImportJob.java:187)
	at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:94)
	at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259)
	at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:668)
	at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
	at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:196)
	at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:176)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
	at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.kitesdk.data.mapreduce.DatasetKeyOutputFormat
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	... 34 more

 

Job.properties

 

# Workflow Parameters
oozie.use.system.libpath=true

 

Output of -shareliblist sqoop

 

[cloudera@quickstart ~]$ oozie admin -shareliblist sqoop
[Available ShareLib]
sqoop
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/ST4-4.0.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/activation-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/ant-1.8.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/ant-launcher-1.8.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/antlr-2.7.7.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/antlr-runtime-3.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/apache-curator-2.6.0.pom
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/apache-log4j-extras-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/asm-3.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/asm-commons-3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/asm-tree-3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/avro-ipc-tests.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/avro-ipc.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/avro-mapred-hadoop2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/bonecp-0.7.1.RELEASE.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/calcite-avatica-1.0.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/calcite-core-1.0.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/calcite-linq4j-1.0.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-compiler-2.7.6.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-compress-1.4.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-dbcp-1.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-io-2.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-jexl-2.1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/commons-pool-1.5.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/curator-client-2.7.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/curator-framework-2.6.0.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/datanucleus-api-jdo-3.2.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/datanucleus-core-3.2.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/datanucleus-rdbms-3.2.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/eigenbase-properties-1.1.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/findbugs-annotations-1.3.9-1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/geronimo-annotation_1.0_spec-1.1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/geronimo-jaspic_1.0_spec-1.0.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/geronimo-jta_1.1_spec-1.1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/groovy-all-2.1.6.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/guava-11.0.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hbase-annotations.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hbase-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-ant.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-cli.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-exec.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-hcatalog-core.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-metastore.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-serde.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-service.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims-0.23.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims-scheduler.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hive-shims.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/hsqldb-1.8.0.10.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/htrace-core-3.1.0-incubating.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/httpclient-4.2.5.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/httpcore-4.2.5.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jackson-annotations-2.3.0.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jackson-core-2.3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jackson-databind-2.3.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/janino-2.7.6.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jdo-api-3.0.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jersey-servlet-1.14.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jetty-all-7.6.0.v20120127.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jline-2.11.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jpam-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jsr305-1.3.9.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/jta-1.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/kite-data-core.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/kite-hadoop-compatibility.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/libfb303-0.9.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/libthrift-0.9.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/logredactor-1.0.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/mail-1.4.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/netty-3.4.0.Final.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/oozie-sharelib-sqoop-4.1.0-cdh5.4.2.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/opencsv-2.3.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-avro.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-column.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-common.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-encoding.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-format.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-generator.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-hadoop-bundle.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-hadoop.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/parquet-jackson.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/snappy-java-1.0.4.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/sqoop.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/stringtemplate-3.2.1.jar
hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20150609033900/sqoop/xz-1.0.jar 

 

6,594 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank abe author-rank
Expert Contributor

Created ‎07-28-2015 02:13 PM

It looks like "kite-data-mapreduce.jar" is missing from the Oozie sharelib. We're tracking this internally. Until then, could you upload this to the sharelib or add it to the Sqoop action as a "file"?

View solution in original post

6,574 Views
0 Kudos
3 REPLIES 3

avatar
author-rank abe author-rank
Expert Contributor

Created ‎07-28-2015 02:13 PM

It looks like "kite-data-mapreduce.jar" is missing from the Oozie sharelib. We're tracking this internally. Until then, could you upload this to the sharelib or add it to the Sqoop action as a "file"?
6,575 Views
0 Kudos

avatar
author-rank Justen
Explorer

Created ‎07-28-2015 02:19 PM

Which version of this library should be used? sharelib doesn't have version numbers for these jars.
6,572 Views
0 Kudos

avatar
author-rank abe author-rank
Expert Contributor

Created ‎07-28-2015 02:23 PM

What ever is available in your CDH distribution.

If you're using parcels, take a look at /opt/cloudera/parcles/CDH/jars.

If you're using packages, take a look at /usr/lib/kite.

You can upload that jar to any directory in HDFS and make it available to your workflow by including it using the "file" element (https://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a3.2.2.1_Adding_Files_and_Archives_f... That would probably be the easiest way to test.
6,569 Views
0 Kudos
Announcements
What's New @ Cloudera
[RELEASED] Cloudera Streaming Analytics 1.14 for Cloudera Pu...
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
View More Announcements