Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Capacity Scheduler Job Hung

Capacity Scheduler Job Hung

Contributor

At my client location they have turned on capacity scheduler , oozie job was initiated, first part it sqoops the data then a pig job is run. Sqoop job was successful, then pig job is initiated but without 0% progress, any pointers ?

3 REPLIES 3
Highlighted

Re: Capacity Scheduler Job Hung

PErhaps Memory issues? I.e. only the AM is started but no room for tasks? Or in other words how much RAM do you have in your queue and how much is

oozie launcher am

oozie launcher mapper

pig am

together using. If there is no space anymore for any other mapper it will wait forever. But you can see this in Resourcemanager. ( if the pig task has mappers scheduled ). Then its a different problem.

Highlighted

Re: Capacity Scheduler Job Hung

Contributor

I checked the memory, it doesn't seem to be a problem. same job runs fine with Fair Scheduler. Is this could be an issue with HA ? I went thru the logs, state is updated from time to time launched to running, then it keeps on looping from that point.

Highlighted

Re: Capacity Scheduler Job Hung

Mentor

check the individual nodemanager logs where tasks fail. I would go through all Capacity Scheduler properties again and make sure FairScheduler policies do not conflict with it. @kjilla

Don't have an account?
Coming from Hortonworks? Activate your account here