Created on 03-28-2020 08:09 PM - last edited on 03-29-2020 08:16 AM by ask_bill_brooks
sqoop import -libjars /app/hadoop_users/Mahfooz/sqoop/mysql-connector-java-5.1.48.jar --connect "jdbc:mysql://wwwwwwwww01.xxxx.example.com:3306/move?verifyServerCertificate=false" --username "root" --password "root" --table cities --fields-terminated-by "," --hive-import --create-hive-table --hive-table test.cities_mysql -- --schema "move"
Warning: /app/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/03/29 03:07:51 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.1
20/03/29 03:07:52 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/03/29 03:07:52 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/03/29 03:07:52 INFO tool.CodeGenTool: Beginning code generation
20/03/29 03:07:52 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:874)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:762)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:785)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:288)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:259)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:245)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:333)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Created 03-29-2020 06:05 AM
@mahfooz The mysql jar should be in the class path for the sqoop client. Also make sure the file ha the correct permissions.
Created on 03-29-2020 10:38 AM - last edited on 03-29-2020 04:22 PM by ask_bill_brooks
Yes, by adding jar in classpath, i am able to import without using -libjars option also as given below.
export HADOOP_CLASSPATH=/app/hadoop_users/Mahfooz/sqoop/mysql-connector-java-5.1.48.jar
sqoop import \
--connect "jdbc:mysql://localhost:3306/move?verifyServerCertificate=false&zeroDateTimeBehavior=round" \
--username "something" \
--password "something" \
--delete-target-dir \
--table users \
--fields-terminated-by "," \
--hive-import \
--create-hive-table \
--hive-table test.users \
-- \
--schema "move"
But what is the use of this -libjars option. I am still confused.