Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Configuring number of concurrently running apps in YARN

avatar
Explorer

Hi, I have observed that for our cluster, running on 7 nodes, with 6 data nodes, at any given point in time no more than 8 apps are running in YARN.

 

When we submit more than 8 apps (MR jobs) concurrently then the apps are in PENDING state. Yarn cluster still has vcores available (idle) and there is free RAM.

 

I could not figure out which configuration setting will enable me to run more apps concurrently in YARN against the default queue, please help.

 

Nitin

1 ACCEPTED SOLUTION

avatar
Explorer

Hrsh, I was able to find the property and I modified

 

queueMaxAppsDefault

 

property and now I get more than 8 apps running concurently.

 

Thanks for your help

 

Nitin

 

 

 

View solution in original post

3 REPLIES 3

avatar
Mentor
Have you looked at this possibly related thread and its resolution? http://community.cloudera.com/t5/Batch-Processing-and-Workflow/Application-master-gets-stuck-when-it... -- Is your situation different?

avatar
Explorer

Thanks for the response Harsh.

 

Unfortunately I could not find property queueMaxAMShareDefault any where in CM. We are using CDH 5.4.5.

 

Nitin

avatar
Explorer

Hrsh, I was able to find the property and I modified

 

queueMaxAppsDefault

 

property and now I get more than 8 apps running concurently.

 

Thanks for your help

 

Nitin