Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Issue with Yarn jobs (accepted state due to AM container)

Rising Star

Hi,

We have an issue with yarn jobs and the number that can be run concurrently . There is a case that 6 TEZ jobs needs to be run concurrently but they all go to accepted state.

I uploaded the following information for your reference. I tried different configuration but still no luck. Please advise me

hive-site.xml under /usr/hdp/current/hive-server2/conf

Tex-site.xml under /usr/hdp/current/tez-client

Yarn-site.xml and Mapr-site.xml under /usr/hdp/current/hadoop-yarn-resourcemanager/etc/hadoop

Also

Yarn queue manager

Thanks

SJ

1 ACCEPTED SOLUTION

Hi @Sanaz Janbakhsh,

Please try increasing the number of node managers and try running the jobs.

Also look for the parameters under Scheduler section of Yarn (yarn.scheduler.capacity.maximum-applications / yarn.scheduler.capacity.<queue-path>.maximum-application) and (yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent) which will restrict the number of concurrent running and pending apps.

Please look at the link for more info.

https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Thanks,

Aditya

View solution in original post

3 REPLIES 3

Rising Star

Also please find the following attatch

acceptedjob.png

accpeted-application-in-rm-ui.png

Hi @Sanaz Janbakhsh,

Please try increasing the number of node managers and try running the jobs.

Also look for the parameters under Scheduler section of Yarn (yarn.scheduler.capacity.maximum-applications / yarn.scheduler.capacity.<queue-path>.maximum-application) and (yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent) which will restrict the number of concurrent running and pending apps.

Please look at the link for more info.

https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Thanks,

Aditya

Rising Star

Hi Aditya,

It was really helpful. Thanks.