Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

sqoop import not responding

sqoop import not responding

Explorer

sqoop import --connect jdbc:mysql://hostname/ambari1 --username ambari1 --password xxxxx --table emp -m 1
/usr/hdp/3.1.0.0-78/hadoop/libexec/hadoop-functions.sh: line 2363: HADOOP_ORG.APACHE.SQOOP.SQOOP_USER: bad substitution
/usr/hdp/3.1.0.0-78/hadoop/libexec/hadoop-functions.sh: line 2458: HADOOP_ORG.APACHE.SQOOP.SQOOP_OPTS: bad substitution
19/08/29 10:32:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/08/29 10:32:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/08/29 10:32:12 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/08/29 10:32:12 INFO tool.CodeGenTool: Beginning code generation
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
19/08/29 10:32:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp` AS t LIMIT 1
19/08/29 10:32:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp` AS t LIMIT 1
19/08/29 10:32:13 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/3.1.0.0-78/hadoop-mapreduce
Note: /tmp/sqoop-gaian/compile/1b81431022615e95f843761509c64995/emp.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/08/29 10:32:14 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-gaian/compile/1b81431022615e95f843761509c64995/emp.jar
19/08/29 10:32:14 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/08/29 10:32:14 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/08/29 10:32:14 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/08/29 10:32:14 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/08/29 10:32:14 INFO mapreduce.ImportJobBase: Beginning import of emp
19/08/29 10:32:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/08/29 10:32:15 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
19/08/29 10:32:16 INFO client.RMProxy: Connecting to ResourceManager at hostname/192.168.24.32:8050
19/08/29 10:32:16 INFO client.AHSProxy: Connecting to Application History server at hostname/192.168.24.32:10200
19/08/29 10:32:17 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /user/gaian/.staging/job_1566883107133_0006
19/08/29 10:32:24 INFO db.DBInputFormat: Using read commited transaction isolation
19/08/29 10:32:24 INFO mapreduce.JobSubmitter: number of splits:1
19/08/29 10:32:24 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1566883107133_0006
19/08/29 10:32:24 INFO mapreduce.JobSubmitter: Executing with tokens: []
19/08/29 10:32:25 INFO conf.Configuration: found resource resource-types.xml at file:/etc/hadoop/3.1.0.0-78/0/resource-types.xml
19/08/29 10:32:25 INFO impl.YarnClientImpl: Submitted application application_1566883107133_0006
19/08/29 10:32:25 INFO mapreduce.Job: The url to track the job: http://hostname:8088/proxy/application_1566883107133_0006/
19/08/29 10:32:25 INFO mapreduce.Job: Running job: job_1566883107133_0006

 

8 REPLIES 8

Re: sqoop import not responding

Guru
@Manoj690

Can you please give a bit more details on at what stage it is not responding? Did it stuck at last line 'Running job: job_1566883107133_0006'?

This can help to clarify the issue.

Cheers
Eric

Re: sqoop import not responding

Explorer

yes i stuck at last line its never ending.

Re: sqoop import not responding

Guru
Have you checked in RM web UI to see if job job_1566883107133_0006 has been accepted or not? If not, you might not have enough resources and RM might be waiting for that.

Cheers
Eric

Re: sqoop import not responding

Explorer

imag.jpeg

Re: sqoop import not responding

Guru
Yeah, that confirms you are running out of resources. Go to RM web UI's home page to see how many resources are left, you might just have to wait for other jobs to finish first.

Re: sqoop import not responding

Explorer

6 apps are in pending status

Highlighted

Re: sqoop import not responding

Mentor

@Manoj690 

 

I think you are missing the MySQL database port  3306 !  the DB listener process runs on that port.  without the port, sqoop won't know how to connect.

Can you try 
sqoop import --connect jdbc:mysql://hostname:3306/ambari1 --username ambari1 --password xxxxx --table emp -m 1

 

From the Ambari UI you should validate the default port 3306 is configured  and the connection is working or  CLI

HTH

 

 

 

Re: sqoop import not responding

Guru
Have you checked what resources your cluster has from RM's web UI page?

This confirms that you are running out of resources and jobs are just waiting to be allocated resources before they can run.

Cheers
Eric