Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

sqoop export with updateonly mode is getting timeout and failing

sqoop export with updateonly mode is getting timeout and failing

New Contributor

I have executed the following command:
sqoop export
–connect jdbc:mysql://172.31.20.247/retail_db
–username sqoopuser \
-P
–table orders_csm
–export-dir retail_db/orders
–update-key order_id
–update-mode updateonly

Here is output and failing after timeout:
18/10/05 01:49:17 INFO mapreduce.Job: Job job_1537636876515_1990 running in uber mode : false
18/10/05 01:49:17 INFO mapreduce.Job: map 0% reduce 0%
18/10/05 01:49:28 INFO mapreduce.Job: map 1% reduce 0%
18/10/05 01:49:40 INFO mapreduce.Job: map 2% reduce 0%
18/10/05 01:50:56 INFO mapreduce.Job: map 3% reduce 0%
18/10/05 01:52:06 INFO mapreduce.Job: map 4% reduce 0%
18/10/05 01:53:22 INFO mapreduce.Job: map 5% reduce 0%
18/10/05 01:54:32 INFO mapreduce.Job: map 6% reduce 0%
18/10/05 01:55:48 INFO mapreduce.Job: map 7% reduce 0%
18/10/05 01:56:58 INFO mapreduce.Job: map 8% reduce 0%
18/10/05 01:58:14 INFO mapreduce.Job: map 9% reduce 0%
18/10/05 01:59:21 INFO mapreduce.Job: map 10% reduce 0%
18/10/05 02:00:37 INFO mapreduce.Job: map 11% reduce 0%

I am facing the same issue, if I execute the command without --update-mode.

The same issue is observed in the Cloudera Sandbox.

 

Need help, how to resolve this issue.

2 REPLIES 2
Highlighted

Re: sqoop export with updateonly mode is getting timeout and failing

New Contributor

More observations.

 

  • in the prevous example i have used a table with no primary key and it took forever to export using updateonly mode and finally it countered timeout.
  • In the following expample I have used a mysql table with primary key. I took long to complete but it was faster and didn't enocunter failure. 

Can anyone help me to know why the updateonly mode is very slow?

 

 

18/10/05 02:11:20 INFO mapreduce.Job:  map 0% reduce 0%

18/10/05 02:11:31 INFO mapreduce.Job:  map 14% reduce 0%

18/10/05 02:11:34 INFO mapreduce.Job:  map 22% reduce 0%

18/10/05 02:11:37 INFO mapreduce.Job:  map 27% reduce 0%

18/10/05 02:11:40 INFO mapreduce.Job:  map 34% reduce 0%

18/10/05 02:11:43 INFO mapreduce.Job:  map 42% reduce 0%

18/10/05 02:11:46 INFO mapreduce.Job:  map 47% reduce 0%

18/10/05 02:11:47 INFO mapreduce.Job:  map 49% reduce 0%

18/10/05 02:11:49 INFO mapreduce.Job:  map 54% reduce 0%

18/10/05 02:11:50 INFO mapreduce.Job:  map 57% reduce 0%

18/10/05 02:11:52 INFO mapreduce.Job:  map 58% reduce 0%

18/10/05 02:11:53 INFO mapreduce.Job:  map 61% reduce 0%

18/10/05 02:11:56 INFO mapreduce.Job:  map 65% reduce 0%

18/10/05 02:11:59 INFO mapreduce.Job:  map 70% reduce 0%

18/10/05 02:12:02 INFO mapreduce.Job:  map 74% reduce 0%

18/10/05 02:12:05 INFO mapreduce.Job:  map 78% reduce 0%

18/10/05 02:12:08 INFO mapreduce.Job:  map 82% reduce 0%

18/10/05 02:12:11 INFO mapreduce.Job:  map 86% reduce 0%

18/10/05 02:12:14 INFO mapreduce.Job:  map 89% reduce 0%

18/10/05 02:12:17 INFO mapreduce.Job:  map 91% reduce 0%

18/10/05 02:12:20 INFO mapreduce.Job:  map 92% reduce 0%

18/10/05 02:12:23 INFO mapreduce.Job:  map 93% reduce 0%

18/10/05 02:12:27 INFO mapreduce.Job:  map 94% reduce 0%

18/10/05 02:12:31 INFO mapreduce.Job:  map 95% reduce 0%

18/10/05 02:12:34 INFO mapreduce.Job:  map 96% reduce 0%

18/10/05 02:12:37 INFO mapreduce.Job:  map 97% reduce 0%

18/10/05 02:12:40 INFO mapreduce.Job:  map 98% reduce 0%

18/10/05 02:12:43 INFO mapreduce.Job:  map 99% reduce 0%

18/10/05 02:12:45 INFO mapreduce.Job:  map 100% reduce 0%

18/10/05 02:12:46 INFO mapreduce.Job: Job job_1537636876515_1992 completed successfully

Re: sqoop export with updateonly mode is getting timeout and failing

Rising Star

Hi,

 

It looks like it has taken only 1 min to make it successful.

 

Started:- 18/10/05 02:11:20 INFO mapreduce.Job:  map 0% reduce 0%

Ended:- 18/10/05 02:12:46 INFO mapreduce.Job: Job job_1537636876515_1992 completed successfully

 

Can you please confirm if this job has run more faster in past?


Regards

Nitish