question Re: sqoop import error in Support Questions

sqoop import error

krol — Fri, 09 Mar 2018 07:02:19 GMT

I am trying to import a table named 'widgets' in the data base 'Testdb' with the command below:

$sqoop import --connect jdbc:mysql://hadoop/Testdb --table widgets -m 4 -username root -P

I get the error "Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver"

Re: sqoop import error

Shelton — Fri, 09 Mar 2018 07:13:28 GMT

@Aymen Rahal

Did you copy the JDBC driver (mysql-connector-java*) to the $SQOOP_HOME/lib directory of your Sqoop installation.?

Then re-run the command

Re: sqoop import error

jsensharma — Fri, 09 Mar 2018 07:17:34 GMT

@Aymen Rahal

You will need to download the mysql-connector-java (MySQL JDBC driver) and then put it to the following location:

 # cp -f /usr/share/java/mysql-connector-java.jar  /usr/hdp/current/sqoop-client/lib/

The you can run the Sqoop command something like following:

 # su -l sqoop -c "sqoop import --connect jdbc:mysql://hadoop/Testdb --table widgets -m 4 --username root --password CHANGEME --driver com.mysql.jdbc.Driver"

Re: sqoop import error

jsensharma — Fri, 09 Mar 2018 07:19:43 GMT

@Aymen Rahal

Following link talks about it in more detail: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_data-movement-and-integration/content/apache_sqoop_connectors.html

Re: sqoop import error

krol — Fri, 09 Mar 2018 07:24:11 GMT

$ls -ltr /usr/share/java/mysql-connector-java-5.1.17.jar

ls: cannot access '/usr/share/java/mysql-connector-java-5.1.17.jar': No such file or directory

Re: sqoop import error

jsensharma — Fri, 09 Mar 2018 07:26:38 GMT

@Aymen Rahal

If you do not have the mysql driver in your host then please install it as following:

# yum install mysql-connector-java*
# cp -f /usr/share/java/mysql-connector-java.jar  /usr/hdp/current/sqoop-client/lib/

If you do not want to use "yum" to install mysql-connector-java then you can also download the tar archive of the connector from the following site and then put the extracted JAR (mysql-connector-java*.jar) inside the "usr/hdp/current/sqoop-client/lib/"

https://dev.mysql.com/downloads/connector/j/

Re: sqoop import error

krol — Fri, 09 Mar 2018 07:38:12 GMT

$ cp -f /usr/share/java/mysql-connector-java.jar /usr/hdp/current/sqoop-client/lib/

cp: '/usr/share/java/mysql-connector-java.jar' and '/usr/hdp/current/sqoop-client/lib/mysql-connector-java.jar' are the same file

Re: sqoop import error

jsensharma — Fri, 09 Mar 2018 07:48:02 GMT

@Aymen Rahal

Do you see the following symlink exist with the following user permission:

# ls -l /usr/hdp/current/sqoop-client/lib/mysql-connector-java.jar
lrwxrwxrwx. 1 root root 40 Jun  9  2017 /usr/hdp/current/sqoop-client/lib/mysql-connector-java.jar -> /usr/share/java/mysql-connector-java.ja

Also do you see the following command shows the class found?

# jar -tvf /usr/share/java/mysql-connector-java.jar | grep 'com.mysql.jdbc.Driver' 
   919 Tue Jan 01 00:00:00 UTC 1980 com/mysql/jdbc/Driver.class

Re: sqoop import error

krol — Fri, 09 Mar 2018 07:48:39 GMT

#su -l sqoop -c "sqoop import --connect jdbc:mysql://hadoop/Testdb --table widgets -m 4 --username root --password CHANGEME --driver com.mysql.jdbc.Driver"

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Re: sqoop import error

jsensharma — Fri, 09 Mar 2018 07:50:15 GMT

Now the error says that it is not able to communicate with MySQL server.

So please check if your mysql server is running file and listening to "hadoop:3306" port

# netstat -tnlpa | grep 3306
# hostname -f
# ps -ef | grep mysql

Also please check if there is any issue/error logged in mysql log.

Re: sqoop import error

krol — Fri, 09 Mar 2018 07:50:38 GMT

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Re: sqoop import error

krol — Fri, 09 Mar 2018 07:55:22 GMT

everything is ok, may be i have to add the port to jdbc:mysql://hadoop:3306 right ?

Re: sqoop import error

jsensharma — Fri, 09 Mar 2018 07:58:45 GMT

@Aymen Rahal

Yes 3306 is the default port for MySQL.

So better to add it in the URL.

# sqoop import --connect jdbc:mysql://hadoop:3306/Testdb --table widgets -m 4 --username root --password CHANGEME --driver com.mysql.jdbc.Driver<br>

Also please check if you are able to login to MySQL using command line or not?

# mysql -u root -p

Re: sqoop import error

krol — Fri, 09 Mar 2018 08:04:24 GMT

I get this error:

18/03/09 01:01:51 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1659) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.main(Sqoop.java:243)

Re: sqoop import error

jsensharma — Fri, 09 Mar 2018 08:05:55 GMT

@Aymen Rahal

The current error is completely different from the one which was mentioned in the original thread so it might confuse the users who hit the same issue described in the Original thread.

Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter at org.apache.sqoop.orm.ClassWriter.generat

Hence i will request you to please mark this HCC thread as "answered" by clicking on the "Accept" button and then open a new thread with the current error. Thats helps many community users to quickly find one specific error with one specific answer.