Created on 05-09-2019 03:07 AM - edited 09-16-2022 07:22 AM
I run the following sqoop merge import
sudo sqoop import \ --connect 'jdbc:sqlserver://1.1.1.1\test_server;database=Training' \ --username Training_user --password Training_user \ --table BigDataTest -m 1 \ --check-column lastmodified \ --merge-key id \ --incremental lastmodified \ --compression-codec=snappy \ --as-parquetfile \ --target-dir /user/hive/warehouse \ --hive-table bigDataTest \ --last-value '2019-05-06 15:07:49.917'
I get this
.
.
.
19/05/09 11:00:50 INFO tool.ImportTool: Final destination exists, will run merge job. 19/05/09 11:00:50 ERROR tool.ImportTool: Import failed: java.io.IOException: Could not load jar /tmp/sqoop-root/compile/e913f7c459cf4e1cdb8a8d5802f1dac2/codegen_BigDataTest.jar into JVM. (Cou
ld not find class BigDataTest.) at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:92) at com.cloudera.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:36) at org.apache.sqoop.tool.ImportTool.loadJars(ImportTool.java:120) at org.apache.sqoop.tool.ImportTool.lastModifiedMerge(ImportTool.java:456) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:522) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) Caused by: java.lang.ClassNotFoundException: BigDataTest at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:270) at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:88) ... 11 more
why? and how I can fix it ?
Created 06-04-2019 01:29 AM
I found this resolution elsewhere:
The problem is resolved after i copied the .class file from /tmp/sqoop-hduser/compile/ to hdfs /home/hduser/ and also the current working directory from where i am running sqoop.
In case that does not work, this should help you get moving:
Specify the --bindir where the compiled code and .jar file should be located.
Without these arguments, Sqoop would place the generated Java source file in your current working directory and the compiled .class file and .jar file in /tmp/sqoop-<username>/compile.
With an example:
sqoop import --bindir ./ --connect jdbc:mysql://localhost/hadoopguide --table widgets