Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Sqoop list-databases requires --connection-manager

avatar
Expert Contributor

I'm using Hortonworks Sandbox HDP 2.4 and I can't use the `list-databases` tool, although `import` is working.

sqoop list-databases --connect jdbc:mysql://<ip>:<port>/ --driver com.mysql.jdbc.Driver --username=myusername -P

This the result that I get

    16/08/11 07:44:47 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
    16/08/11 07:44:47 INFO manager.SqlManager: Using default fetchSize of 1000
    16/08/11 07:44:47 ERROR manager.SqlManager: Generic SqlManager.listDatabases() not implemented.
    Could not retrieve database list from server
    16/08/11 07:44:47 ERROR tool.ListDatabasesTool: manager.listDatabases() returned null 

The response says that I need to set the `--connection-manager` parameter, which I do not know. If I will take out the --driver argument, I get exception:

16/08/11 07:48:12 ERROR manager.CatalogQueryManager: Failed to list databases
    com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
            at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
            at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:344)
            at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2336)
            at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370)
            at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154)
            at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
            at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
            at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
            at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
            at java.sql.DriverManager.getConnection(DriverManager.java:571)
            at java.sql.DriverManager.getConnection(DriverManager.java:215)
            at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:885)
            at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
            at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
            at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
            at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
            at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
            at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
            at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
            at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
    Caused by: java.net.ConnectException: Connection refused
            at java.net.PlainSocketImpl.socketConnect(Native Method)
            at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
            at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
            at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
            at java.net.Socket.connect(Socket.java:579)
            at java.net.Socket.connect(Socket.java:528)
            at java.net.Socket.<init>(Socket.java:425)
            at java.net.Socket.<init>(Socket.java:241)
            at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:258)
            at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:295)
            ... 24 more
    16/08/11 07:48:12 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
    java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
            at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73)
            at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
            at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
            at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
            at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
            at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
            at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
            at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
            at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:344)
            at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2336)
            at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370)
            at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154)
            at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
            at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
            at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
            at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
            at java.sql.DriverManager.getConnection(DriverManager.java:571)
            at java.sql.DriverManager.getConnection(DriverManager.java:215)
            at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:885)
            at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
            at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
            ... 7 more
    Caused by: java.net.ConnectException: Connection refused
            at java.net.PlainSocketImpl.socketConnect(Native Method)
            at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
            at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
            at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
            at java.net.Socket.connect(Socket.java:579)
            at java.net.Socket.connect(Socket.java:528)
            at java.net.Socket.<init>(Socket.java:425)
            at java.net.Socket.<init>(Socket.java:241)
            at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:258)
            at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:295)
            ... 24 more 

What's the value for `--connection-manager`?

1 ACCEPTED SOLUTION

avatar
Expert Contributor

What worked for me is this:

$ sqoop list-databases 
	--connect jdbc:mysql://<ip>:<port>/ 
	--connection-manager org.apache.sqoop.manager.MySQLManager
	--username=myusername -P

Same thing goes for list-tables tool

$ sqoop list-tables 
	--connect jdbc:mysql://<ip>:<port>/mydb
	--connection-manager org.apache.sqoop.manager.MySQLManager
	--username=myusername -P

I removed the --driver flag.

View solution in original post

4 REPLIES 4

avatar
@J. D. Bacolod

Can you try something like below?

sqoop list-databases --connect jdbc:mysql://<mysqlserver>/ --username root

avatar
Expert Contributor

I've got the same result as the last exception.

avatar
Expert Contributor

Can you try as below it worked for me.

$sqoop list-databases \ --connect jdbc:mysql://dbhost/ \ --username root

avatar
Expert Contributor

What worked for me is this:

$ sqoop list-databases 
	--connect jdbc:mysql://<ip>:<port>/ 
	--connection-manager org.apache.sqoop.manager.MySQLManager
	--username=myusername -P

Same thing goes for list-tables tool

$ sqoop list-tables 
	--connect jdbc:mysql://<ip>:<port>/mydb
	--connection-manager org.apache.sqoop.manager.MySQLManager
	--username=myusername -P

I removed the --driver flag.