Exercise 1 Sqoop issue

[#1]

Hi Cloudera team, I am new to sqoop. Downloaded CDH 5.10 and using Sqoop version 1.4.6. While trying to import the table into hive using the below sqoop import command, it got hanged after "Will generate java class as codegen_departments" message. Below is the log in verbose mode. Could you please help me to understand on the issue.

[cloudera@quickstart ~]$ sqoop import -m 2 --connect jdbc:mysql://quickstart.cloudera:8020/retail_db --username=retail_dba --password=cloudera --table departments --compression-codec=snappy --as-parquetfile --warehouse-dir=/user/hive/warehouse/sriram/ --hive-import --verbose Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 17/08/02 11:18:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.10.0 17/08/02 11:18:38 DEBUG tool.BaseSqoopTool: Enabled debug logging. 17/08/02 11:18:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 17/08/02 11:18:38 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override 17/08/02 11:18:38 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc. 17/08/02 11:18:38 WARN tool.BaseSqoopTool: It seems that you're doing hive import directly into default 17/08/02 11:18:38 WARN tool.BaseSqoopTool: hive warehouse directory which is not supported. Sqoop is 17/08/02 11:18:38 WARN tool.BaseSqoopTool: firstly importing data into separate directory and then 17/08/02 11:18:38 WARN tool.BaseSqoopTool: inserting data into hive. Please consider removing 17/08/02 11:18:38 WARN tool.BaseSqoopTool: --target-dir or --warehouse-dir into /user/hive/warehouse in 17/08/02 11:18:38 WARN tool.BaseSqoopTool: case that you will detect any issues. 17/08/02 11:18:38 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory 17/08/02 11:18:38 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 17/08/02 11:18:38 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory 17/08/02 11:18:38 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop! 17/08/02 11:18:38 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory 17/08/02 11:18:38 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql: 17/08/02 11:18:38 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 17/08/02 11:18:38 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.MySQLManager@61b1c3a3 17/08/02 11:18:38 INFO tool.CodeGenTool: Beginning code generation 17/08/02 11:18:38 INFO tool.CodeGenTool: Will generate java class as codegen_departments 17/08/02 11:18:38 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM `departments` AS t LIMIT 1 17/08/02 11:18:39 DEBUG manager.SqlManager: No connection paramenters specified. Using regular API for making connection.

[#2]

Hi, Could anyone please help me on the above issue.

[#3]

Hi Sriraaman,

The problem is the port for mysql quickstart db is 3306 and not 8020.

It worked when I changed the port to 3306.

Regards,

Meher

[#4]

Just remove the port number , try the below command as a test  . 

sqoop list-tables --connect jdbc:mysql://quickstart.cloudera/retail_db --username retail_dba --password clouderaF

I tested it - it worked 

For your sqoop command . perform the below mate

 sqoop import -m 2 --connect jdbc:mysql://quickstart.cloudera/retail_db 
--username=retail_dba --password=cloudera --table departments
 --compression-codec=snappy --as-parquetfile --warehouse-dir=/user/hive/warehouse/sriram/ --hive-import

 Let me know if that works . 

[#5 · Accepted solution]

[#6]

Thanks Guna. It is working. But could you please tell me why the command i have given is not working. If am correct, default port of mysql is 3306 right? In fact, I tried logging into mysql using the below command and it works fine.

mysql --host=127.0.0.1 --port=3306 --user=retail_dba --password=cloudera

Thanks,

Sriram

[#7]

As I specified in my earlier post just change port 8020 to 3306 which you specified in your first post.

It should work.

[#8]

Sqoop does not come with the JDBC driver , as you already knew that sqoop is just a client tool it uses the JDBC driver to connect to data source in our case its MYSQL , if you look under the directory 

/var/lib/sqoop/

should have a MYSQL jdbc driver , internally it will take care of all the connections ,query execution , result sets etc . Since we are using localhost it will connect the locally runining server , if you want to access the remote server then we use 3306 , however you can also mention the port number when refering to localhost , i dont have the vm to see their socket file configuration . moreover you have to enforce the TCP/IP connection because by default when we use localhost it will refer the socket file . Hope this helps .