Created on 01-04-2016 10:57 PM - edited 09-16-2022 02:55 AM
I am following the steps in Execrice 1 . sqoop is failing. Below is more details.
$ sqoop import-all-tables -m 1 --connect jdbc:mysql://{{cluster_data.manager_node_hostname}}:3306/retail_db --username=retail_dba --password=cloudera --compression-codec=snappy --as-parquetfile --warehouse-dir=/user/hive/warehouse --hive-import
Warning: /opt/cloudera/parcels/CDH-5.4.2-1.cdh5.4.2.p0.2/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/05 01:45:42 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.4.2
16/01/05 01:45:42 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/01/05 01:45:42 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
16/01/05 01:45:42 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
16/01/05 01:45:42 WARN tool.BaseSqoopTool: It seems that you're doing hive import directly into default
16/01/05 01:45:42 WARN tool.BaseSqoopTool: hive warehouse directory which is not supported. Sqoop is
16/01/05 01:45:42 WARN tool.BaseSqoopTool: firstly importing data into separate directory and then
16/01/05 01:45:42 WARN tool.BaseSqoopTool: inserting data into hive. Please consider removing
16/01/05 01:45:42 WARN tool.BaseSqoopTool: --target-dir or --warehouse-dir into /user/hive/warehouse in
16/01/05 01:45:42 WARN tool.BaseSqoopTool: case that you will detect any issues.
16/01/05 01:45:42 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
16/01/05 01:45:43 ERROR manager.CatalogQueryManager: Failed to list tables
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
....
....
....
Caused by: java.net.UnknownHostException: {{cluster_data.manager_node_hostname}}
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
at java.net.InetAddress.getAllByName(InetAddress.java:1162)
at java.net.InetAddress.getAllByName(InetAddress.java:1098)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:190)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:297)
... 24 more
I am using -m 1 based on earlier thread. MySQL instance is running.
Any assistance will be much appreciated.
Created 01-05-2016 08:41 AM
You're supposed to replace {{cluster_data.manager_node_hostname}} with the hostname of the MySQL server, which on Cloudera Live is the Manager Node. If you're using Cloudera Live, you should refer to the copy of the tutorial as linked to in your welcome email. It will resolve all such variables for your specific cluster. The copy on the website is just for informational purposes - the procedure assumes a lot about the cluster, so you should really just use the copy that's on your cluster.
Created 01-05-2016 08:41 AM
You're supposed to replace {{cluster_data.manager_node_hostname}} with the hostname of the MySQL server, which on Cloudera Live is the Manager Node. If you're using Cloudera Live, you should refer to the copy of the tutorial as linked to in your welcome email. It will resolve all such variables for your specific cluster. The copy on the website is just for informational purposes - the procedure assumes a lot about the cluster, so you should really just use the copy that's on your cluster.
Created 12-10-2016 01:30 AM
I am using cloudera express so mysql is in quickstart.cloudera. But i keep getting Retrying to connect to server error. mysql is present uncer /var/lib/mysql. I have given sudo chmod 777 /var/lib/mysql as well but no luck.
Any idea what is wrong here.
mysql gives access denied error for user cloudera@localhost using password no
Created 05-16-2017 07:03 PM
Hello Sean,
I am using cloudera quickstart VM (5.10) to list the databases: but facing Communications link failure issue any help would be appreciated.
Command line:
sqoop list-databases \
--connect "jdbc:mysql://quickstart.cloudera:7180" \
--username retail_dba \
--password cloudera
ERROR:
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/05/16 15:40:17 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.10.0
17/05/16 15:40:17 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/05/16 15:40:18 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
17/05/16 15:43:38 ERROR manager.CatalogQueryManager: Failed to list databases
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:627)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1013)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2234)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064)
at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:790)
at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:44)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:395)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:325)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:904)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2914)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:559)
... 25 more
17/05/16 15:43:38 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73)
at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:627)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1013)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2234)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064)
at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:790)
at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:44)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:395)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:325)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:904)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
... 7 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2914)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:559)
... 25 more