Support Questions
Find answers, ask questions, and share your expertise

what determines how many jobs you can run in parallel in YARN

Explorer

With just the default queue..we could run 4 jobs in parallel.

Adding an extra queue did not allow us to run any more than 4 jobs..

we still were unable to run extra jobs in parallel. we can use the new queue but cannot get above 4 jobs running and the rest sit in ACCEPTED mode waiting on a job to stop.

1 REPLY 1

Super Guru

@Olivia Smith

There can be multiple reasons why this happening. I would say, try to run 5th job and when it gets stuck in ACCEPTED state, please check resource manager logs(grep for application ID of 5th job) to see what is happening.

You can check more details on capacity scheduler related configuration here - https://community.hortonworks.com/questions/187933/unable-to-run-the-hadoop-commands-in-sandbox.html...

; ;