Support Questions
Find answers, ask questions, and share your expertise

Issue with Yarn jobs (accepted state due to AM container)

Rising Star

Hi,

We have an issue with yarn jobs and the number that can be run concurrently . There is a case that 6 TEZ jobs needs to be run concurrently but they all go to accepted state.

I uploaded the following information for your reference. I tried different configuration but still no luck. Please advise me

hive-site.xml under /usr/hdp/current/hive-server2/conf

Tex-site.xml under /usr/hdp/current/tez-client

Yarn-site.xml and Mapr-site.xml under /usr/hdp/current/hadoop-yarn-resourcemanager/etc/hadoop

Also

Yarn queue manager

Thanks

SJ

1 ACCEPTED SOLUTION

Accepted Solutions

Hi @Sanaz Janbakhsh,

Please try increasing the number of node managers and try running the jobs.

Also look for the parameters under Scheduler section of Yarn (yarn.scheduler.capacity.maximum-applications / yarn.scheduler.capacity.<queue-path>.maximum-application) and (yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent) which will restrict the number of concurrent running and pending apps.

Please look at the link for more info.

https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Thanks,

Aditya

View solution in original post

3 REPLIES 3

Rising Star

Also please find the following attatch

acceptedjob.png

accpeted-application-in-rm-ui.png

Hi @Sanaz Janbakhsh,

Please try increasing the number of node managers and try running the jobs.

Also look for the parameters under Scheduler section of Yarn (yarn.scheduler.capacity.maximum-applications / yarn.scheduler.capacity.<queue-path>.maximum-application) and (yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent) which will restrict the number of concurrent running and pending apps.

Please look at the link for more info.

https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Thanks,

Aditya

View solution in original post

Rising Star

Hi Aditya,

It was really helpful. Thanks.