Reply
baj
New Contributor
Posts: 6
Registered: ‎06-16-2014

sqoop job is stuck at 5%

I have a pseudo distributed hadoop cluster running CDH5.0.2. I'm running a sqoop import command:

sudo -u sqoop sqoop import --connect "jdbc:sqlserver://x.x.x.x:1433;databaseName=yyyyy" --username x --password y --table table_name

I'm just importing a very small table that has 12 rows and 2 columns for testing. The job has been running for half hour. On My resource manager, the mapper tasks' status are listed as NEW and their state are listed as SCHEDULED. I don't think it ever runs!

When i list the job on yarn using:

yarn application -list

i'm getting:

 

14/07/01 15:55:06 INFO client.RMProxy: Connecting to ResourceManager at host/x.x.x.x:8032
Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):1
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
application_1404252440376_0001 ActivityType.jar MAPREDUCE sqoop root.sqoop RUNNING UNDEFINED 5% http://host:42583

 

 

this is the Application Master log im seeing. How do i fix this?

 

2014-07-01 15:14:12,880 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-07-01 15:14:12,885 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-07-01 15:14:12,888 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at host/x.x.x.x:8030
2014-07-01 15:14:12,973 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: maxContainerCapability: 1024
2014-07-01 15:14:12,973 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: root.sqoop
2014-07-01 15:14:12,977 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2014-07-01 15:14:12,979 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies : 500
2014-07-01 15:14:12,985 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1404252440376_0001Job Transitioned from INITED to SETUP
2014-07-01 15:14:12,987 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2014-07-01 15:14:12,997 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1404252440376_0001Job Transitioned from SETUP to RUNNING
2014-07-01 15:14:13,018 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1404252440376_0001_m_000000 Task Transitioned from NEW to SCHEDULED
2014-07-01 15:14:13,019 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1404252440376_0001_m_000001 Task Transitioned from NEW to SCHEDULED
2014-07-01 15:14:13,019 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1404252440376_0001_m_000002 Task Transitioned from NEW to SCHEDULED
2014-07-01 15:14:13,019 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1404252440376_0001_m_000003 Task Transitioned from NEW to SCHEDULED
2014-07-01 15:14:13,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1404252440376_0001_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-07-01 15:14:13,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1404252440376_0001_m_000001_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-07-01 15:14:13,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1404252440376_0001_m_000002_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-07-01 15:14:13,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1404252440376_0001_m_000003_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-07-01 15:14:13,022 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceReqt:1024
2014-07-01 15:14:13,066 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1404252440376_0001, File: hdfs://host:8020/user/sqoop/.staging/job_1404252440376_0001/job_1404252440376_0001_1.jhist
2014-07-01 15:14:13,976 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:4 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2014-07-01 15:14:14,054 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1404252440376_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:0, vCores:0> knownNMs=1

 

 

Cloudera Employee abe
Cloudera Employee
Posts: 109
Registered: ‎08-08-2013

Re: sqoop job is stuck at 5%

Hey there,

Which scheduler are you using? Have you try restarting your YARN cluster?

-Abe
Highlighted
baj
New Contributor
Posts: 6
Registered: ‎06-16-2014

Re: sqoop job is stuck at 5%

i ended up reinstalling the product and it works fine now. Not really sure what went wrong.

 

thanks for your reply. 

Cloudera Employee abe
Cloudera Employee
Posts: 109
Registered: ‎08-08-2013

Re: sqoop job is stuck at 5%

Glad things are working at least. Thanks for getting back to us!
Announcements
New solutions