Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Tasks in pending state

Tasks in pending state

Explorer

Hi

I have HDP 2.6 with RHEL 7.4 ( Single node cluster). At any time my cluster allows only one job to execute. The other job is going to the accepted state until the job completes.

My server is having 16 cores and 64 GB RAM. Attache is my Yarn config.

RM memory and Yarn config are attached for reference.

Do I miss something here? Let me know if you have any suggestions.

I referred the recommendations in below discussion but no luck. My server is new and hard is new ( AWS server).

https://community.hortonworks.com/questions/27334/mapreduce-job-hang-waiting-for-am-container-to-be....

Thx

Muthu

83526-screenshot-161.png

83525-rm-meory.png

4 REPLIES 4

Re: Tasks in pending state

Various scenarios are possible

1. First job is consuming all VCores or Memory hence there is no resporuce left for the next job to spawn.
2. Prempetion has been disabled for the queue.
3. UserLimit is 100% for a given user hence the job is allowed to occupy the complete queue.


Re: Tasks in pending state

Explorer

Hi Gautam

Any suggestions on how to resolve this issue. What properties i need to configure and its optimal value.

In my case it any point, if we can able to run 5 jobs it should be fine. This is Demo server.

Thanks in advance.

Muthu

Re: Tasks in pending state

Various scenarios are possible

1. First job is consuming all VCores or Memory hence there is no resporuce left for the next job to spawn.
2. Prempetion has been disabled for the queue.
3. UserLimit is 100% for a given user hence the job is allowed to occupy the complete queue.


Re: Tasks in pending state

Explorer

Hello All

I resolved this issue by updating the below property.

yarn.scheduler.capacity.maximum-am-resource-percent=0.6

the default value was 0.2 .

Thx

Muthu