sqoop import error

[#1]

I am trying to import a table named 'widgets' in the data base 'Testdb' with the command below:

$sqoop import --connect jdbc:mysql://hadoop/Testdb --table widgets -m 4 -username root -P

I get the error "Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver"

[#2]

@Aymen Rahal

Did you copy the JDBC driver (mysql-connector-java*) to the $SQOOP_HOME/lib directory of your Sqoop installation.?

Then re-run the command

[#3]

$ls -ltr /usr/share/java/mysql-connector-java-5.1.17.jar

ls: cannot access '/usr/share/java/mysql-connector-java-5.1.17.jar': No such file or directory

[#4]

@Aymen Rahal

You will need to download the mysql-connector-java (MySQL JDBC driver) and then put it to the following location:

 # cp -f /usr/share/java/mysql-connector-java.jar  /usr/hdp/current/sqoop-client/lib/

The you can run the Sqoop command something like following:

 # su -l sqoop -c "sqoop import --connect jdbc:mysql://hadoop/Testdb --table widgets -m 4 --username root --password CHANGEME --driver com.mysql.jdbc.Driver"

.

[#5]

@Aymen Rahal

Following link talks about it in more detail: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_data-movement-and-integration/content/ap...

[#6]

$ cp -f /usr/share/java/mysql-connector-java.jar /usr/hdp/current/sqoop-client/lib/

cp: '/usr/share/java/mysql-connector-java.jar' and '/usr/hdp/current/sqoop-client/lib/mysql-connector-java.jar' are the same file

[#7]

@Aymen Rahal

Do you see the following symlink exist with the following user permission:

# ls -l /usr/hdp/current/sqoop-client/lib/mysql-connector-java.jar
lrwxrwxrwx. 1 root root 40 Jun  9  2017 /usr/hdp/current/sqoop-client/lib/mysql-connector-java.jar -> /usr/share/java/mysql-connector-java.ja

.

Also do you see the following command shows the class found?

# jar -tvf /usr/share/java/mysql-connector-java.jar | grep 'com.mysql.jdbc.Driver' 
   919 Tue Jan 01 00:00:00 UTC 1980 com/mysql/jdbc/Driver.class

.

[#8 · Accepted solution]

@Aymen Rahal

If you do not have the mysql driver in your host then please install it as following:

# yum install mysql-connector-java*
# cp -f /usr/share/java/mysql-connector-java.jar  /usr/hdp/current/sqoop-client/lib/

.

If you do not want to use "yum" to install mysql-connector-java then you can also download the tar archive of the connector from the following site and then put the extracted JAR (mysql-connector-java*.jar) inside the "usr/hdp/current/sqoop-client/lib/"

https://dev.mysql.com/downloads/connector/j/

.

[#9]

#su -l sqoop -c "sqoop import --connect jdbc:mysql://hadoop/Testdb --table widgets -m 4 --username root --password CHANGEME --driver com.mysql.jdbc.Driver"

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

[#10]

Now the error says that it is not able to communicate with MySQL server.

So please check if your mysql server is running file and listening to "hadoop:3306" port

# netstat -tnlpa | grep 3306
# hostname -f
# ps -ef | grep mysql

.

Also please check if there is any issue/error logged in mysql log.

.

[#11]

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

[#12]

everything is ok, may be i have to add the port to jdbc:mysql://hadoop:3306 right ?

[#13]

@Aymen Rahal

Yes 3306 is the default port for MySQL.

So better to add it in the URL.

# sqoop import --connect jdbc:mysql://hadoop:3306/Testdb --table widgets -m 4 --username root --password CHANGEME --driver com.mysql.jdbc.Driver<br>

.

Also please check if you are able to login to MySQL using command line or not?

# mysql -u root -p

.

[#14]

I get this error:

18/03/09 01:01:51 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1659) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.main(Sqoop.java:243)

[#15]

@Aymen Rahal

The current error is completely different from the one which was mentioned in the original thread so it might confuse the users who hit the same issue described in the Original thread.

Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter at org.apache.sqoop.orm.ClassWriter.generat

.

Hence i will request you to please mark this HCC thread as "answered" by clicking on the "Accept" button and then open a new thread with the current error. Thats helps many community users to quickly find one specific error with one specific answer.