Tasks in pending state

[#1]

Hi

I have HDP 2.6 with RHEL 7.4 ( Single node cluster). At any time my cluster allows only one job to execute. The other job is going to the accepted state until the job completes.

My server is having 16 cores and 64 GB RAM. Attache is my Yarn config.

RM memory and Yarn config are attached for reference.

Do I miss something here? Let me know if you have any suggestions.

I referred the recommendations in below discussion but no luck. My server is new and hard is new ( AWS server).

https://community.hortonworks.com/questions/27334/mapreduce-job-hang-waiting-for-am-container-to-be....

Thx

Muthu

[#2]

Various scenarios are possible

1. First job is consuming all VCores or Memory hence there is no resporuce left for the next job to spawn.
2. Prempetion has been disabled for the queue.
3. UserLimit is 100% for a given user hence the job is allowed to occupy the complete queue.

[#3]

Hi Gautam

Any suggestions on how to resolve this issue. What properties i need to configure and its optimal value.

In my case it any point, if we can able to run 5 jobs it should be fine. This is Demo server.

Thanks in advance.

Muthu

[#4]

Various scenarios are possible

1. First job is consuming all VCores or Memory hence there is no resporuce left for the next job to spawn.
2. Prempetion has been disabled for the queue.
3. UserLimit is 100% for a given user hence the job is allowed to occupy the complete queue.

[#5]

Hello All

I resolved this issue by updating the below property.

yarn.scheduler.capacity.maximum-am-resource-percent=0.6

the default value was 0.2 .

Thx

Muthu