question Re: Configuring number of concurrently running apps in YARN in Archives of Support Questions (Read Only)

Configuring number of concurrently running apps in YARN

NitinA — Fri, 16 Sep 2022 09:41:05 GMT

Hi, I have observed that for our cluster, running on 7 nodes, with 6 data nodes, at any given point in time no more than 8 apps are running in YARN.

When we submit more than 8 apps (MR jobs) concurrently then the apps are in PENDING state. Yarn cluster still has vcores available (idle) and there is free RAM.

I could not figure out which configuration setting will enable me to run more apps concurrently in YARN against the default queue, please help.

Nitin

Re: Configuring number of concurrently running apps in YARN

Harsh J — Fri, 18 Sep 2015 11:28:10 GMT

Have you looked at this possibly related thread and its resolution? http://community.cloudera.com/t5/Batch-Processing-and-Workflow/Application-master-gets-stuck-when-it-gets-more-then-half/m-p/31393#M1641 -- Is your situation different?

Re: Configuring number of concurrently running apps in YARN

NitinA — Fri, 18 Sep 2015 15:39:07 GMT

Thanks for the response Harsh.

Unfortunately I could not find property queueMaxAMShareDefault any where in CM. We are using CDH 5.4.5.

Nitin

Re: Configuring number of concurrently running apps in YARN

NitinA — Fri, 18 Sep 2015 17:13:11 GMT

Hrsh, I was able to find the property and I modified

queueMaxAppsDefault

property and now I get more than 8 apps running concurently.

Thanks for your help

Nitin