Support Questions

Find answers, ask questions, and share your expertise

number of active containers limited to 160 - cluster is never more than 30% utilized

New Contributor

Hi All,

I have a cluster with 20 nodes with 2 distinct hardware configs so we are using config groups. Our problem is that even though we have available VCores and Mem we can never get more than 160 allocated containers. I hope the screen shot shows my issue - plenty of resources but only 160 allocated containers.

Any pointers would be appreciated.



New Contributor

A little more info.

We are able to get beyond the 160 container limit by running multiple spark-shell with 100 executors each so perhaps this is limited to MR2.