Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

-libjar option is not working in sqoop import for sqoop version 1.4.6

avatar
Explorer

 sqoop import   -libjars /app/hadoop_users/Mahfooz/sqoop/mysql-connector-java-5.1.48.jar   --connect "jdbc:mysql://wwwwwwwww01.xxxx.example.com:3306/move?verifyServerCertificate=false"   --username "root"   --password "root"   --table cities   --fields-terminated-by ","   --hive-import   --create-hive-table   --hive-table test.cities_mysql   --   --schema "move"
Warning: /app/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/03/29 03:07:51 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.1
20/03/29 03:07:52 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/03/29 03:07:52 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/03/29 03:07:52 INFO tool.CodeGenTool: Beginning code generation
20/03/29 03:07:52 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
        at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:874)
        at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:762)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:785)
        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:288)
        at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:259)
        at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:245)
        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:333)
        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858)
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

2 REPLIES 2

avatar
Super Guru

@mahfooz  The mysql jar should be in the class path for the sqoop client.  Also make sure the file ha the correct permissions.

avatar
Explorer

Yes, by adding jar in classpath, i am able to import without using -libjars option also as given below.

 

export HADOOP_CLASSPATH=/app/hadoop_users/Mahfooz/sqoop/mysql-connector-java-5.1.48.jar
sqoop import \
  --connect "jdbc:mysql://localhost:3306/move?verifyServerCertificate=false&zeroDateTimeBehavior=round" \
  --username "something" \
  --password "something" \
  --delete-target-dir \
  --table users \
  --fields-terminated-by "," \
  --hive-import \
  --create-hive-table \
  --hive-table test.users \
  -- \
  --schema "move"

 

 

But what is the use of this -libjars option. I am still confused.