Reply
New Contributor
Posts: 5
Registered: ‎07-26-2018

sqoop export with updateonly mode is getting timeout and failing

I have executed the following command:
sqoop export
–connect jdbc:mysql://172.31.20.247/retail_db
–username sqoopuser \
-P
–table orders_csm
–export-dir retail_db/orders
–update-key order_id
–update-mode updateonly

Here is output and failing after timeout:
18/10/05 01:49:17 INFO mapreduce.Job: Job job_1537636876515_1990 running in uber mode : false
18/10/05 01:49:17 INFO mapreduce.Job: map 0% reduce 0%
18/10/05 01:49:28 INFO mapreduce.Job: map 1% reduce 0%
18/10/05 01:49:40 INFO mapreduce.Job: map 2% reduce 0%
18/10/05 01:50:56 INFO mapreduce.Job: map 3% reduce 0%
18/10/05 01:52:06 INFO mapreduce.Job: map 4% reduce 0%
18/10/05 01:53:22 INFO mapreduce.Job: map 5% reduce 0%
18/10/05 01:54:32 INFO mapreduce.Job: map 6% reduce 0%
18/10/05 01:55:48 INFO mapreduce.Job: map 7% reduce 0%
18/10/05 01:56:58 INFO mapreduce.Job: map 8% reduce 0%
18/10/05 01:58:14 INFO mapreduce.Job: map 9% reduce 0%
18/10/05 01:59:21 INFO mapreduce.Job: map 10% reduce 0%
18/10/05 02:00:37 INFO mapreduce.Job: map 11% reduce 0%

I am facing the same issue, if I execute the command without --update-mode.

The same issue is observed in the Cloudera Sandbox.

 

Need help, how to resolve this issue.

Highlighted
New Contributor
Posts: 5
Registered: ‎07-26-2018

Re: sqoop export with updateonly mode is getting timeout and failing

More observations.

 

  • in the prevous example i have used a table with no primary key and it took forever to export using updateonly mode and finally it countered timeout.
  • In the following expample I have used a mysql table with primary key. I took long to complete but it was faster and didn't enocunter failure. 

Can anyone help me to know why the updateonly mode is very slow?

 

 

18/10/05 02:11:20 INFO mapreduce.Job:  map 0% reduce 0%

18/10/05 02:11:31 INFO mapreduce.Job:  map 14% reduce 0%

18/10/05 02:11:34 INFO mapreduce.Job:  map 22% reduce 0%

18/10/05 02:11:37 INFO mapreduce.Job:  map 27% reduce 0%

18/10/05 02:11:40 INFO mapreduce.Job:  map 34% reduce 0%

18/10/05 02:11:43 INFO mapreduce.Job:  map 42% reduce 0%

18/10/05 02:11:46 INFO mapreduce.Job:  map 47% reduce 0%

18/10/05 02:11:47 INFO mapreduce.Job:  map 49% reduce 0%

18/10/05 02:11:49 INFO mapreduce.Job:  map 54% reduce 0%

18/10/05 02:11:50 INFO mapreduce.Job:  map 57% reduce 0%

18/10/05 02:11:52 INFO mapreduce.Job:  map 58% reduce 0%

18/10/05 02:11:53 INFO mapreduce.Job:  map 61% reduce 0%

18/10/05 02:11:56 INFO mapreduce.Job:  map 65% reduce 0%

18/10/05 02:11:59 INFO mapreduce.Job:  map 70% reduce 0%

18/10/05 02:12:02 INFO mapreduce.Job:  map 74% reduce 0%

18/10/05 02:12:05 INFO mapreduce.Job:  map 78% reduce 0%

18/10/05 02:12:08 INFO mapreduce.Job:  map 82% reduce 0%

18/10/05 02:12:11 INFO mapreduce.Job:  map 86% reduce 0%

18/10/05 02:12:14 INFO mapreduce.Job:  map 89% reduce 0%

18/10/05 02:12:17 INFO mapreduce.Job:  map 91% reduce 0%

18/10/05 02:12:20 INFO mapreduce.Job:  map 92% reduce 0%

18/10/05 02:12:23 INFO mapreduce.Job:  map 93% reduce 0%

18/10/05 02:12:27 INFO mapreduce.Job:  map 94% reduce 0%

18/10/05 02:12:31 INFO mapreduce.Job:  map 95% reduce 0%

18/10/05 02:12:34 INFO mapreduce.Job:  map 96% reduce 0%

18/10/05 02:12:37 INFO mapreduce.Job:  map 97% reduce 0%

18/10/05 02:12:40 INFO mapreduce.Job:  map 98% reduce 0%

18/10/05 02:12:43 INFO mapreduce.Job:  map 99% reduce 0%

18/10/05 02:12:45 INFO mapreduce.Job:  map 100% reduce 0%

18/10/05 02:12:46 INFO mapreduce.Job: Job job_1537636876515_1992 completed successfully

Announcements