Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Issue with Yarn jobs (accepted state due to AM container)

avatar
Expert Contributor

Hi,

We have an issue with yarn jobs and the number that can be run concurrently . There is a case that 6 TEZ jobs needs to be run concurrently but they all go to accepted state.

I uploaded the following information for your reference. I tried different configuration but still no luck. Please advise me

hive-site.xml under /usr/hdp/current/hive-server2/conf

Tex-site.xml under /usr/hdp/current/tez-client

Yarn-site.xml and Mapr-site.xml under /usr/hdp/current/hadoop-yarn-resourcemanager/etc/hadoop

Also

Yarn queue manager

Thanks

SJ

1 ACCEPTED SOLUTION

avatar
Super Guru

Hi @Sanaz Janbakhsh,

Please try increasing the number of node managers and try running the jobs.

Also look for the parameters under Scheduler section of Yarn (yarn.scheduler.capacity.maximum-applications / yarn.scheduler.capacity.<queue-path>.maximum-application) and (yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent) which will restrict the number of concurrent running and pending apps.

Please look at the link for more info.

https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Thanks,

Aditya

View solution in original post

3 REPLIES 3

avatar
Expert Contributor

Also please find the following attatch

acceptedjob.png

accpeted-application-in-rm-ui.png

avatar
Super Guru

Hi @Sanaz Janbakhsh,

Please try increasing the number of node managers and try running the jobs.

Also look for the parameters under Scheduler section of Yarn (yarn.scheduler.capacity.maximum-applications / yarn.scheduler.capacity.<queue-path>.maximum-application) and (yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent) which will restrict the number of concurrent running and pending apps.

Please look at the link for more info.

https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Thanks,

Aditya

avatar
Expert Contributor

Hi Aditya,

It was really helpful. Thanks.