Hi all,

I have a strange issue related to spark ressources allocation in Yarn. The situation is the following: We have a Cloudera cluster with CDH 5.4.2 and Spark 1.3.0. Regarding this issue there are two main users of the cluster. Thereby the following scheduling issues using Yarn occurs:

* When user A runs a couple of spark job, user B has no problems to start additional spark jobs and allocate corresponding ressources

* But when user B has at least one spark job running, user A is not able to start any other job, although only half of the cluster ressources are used

* Both are running under the default pool, just different users

So, my questions would be:

* Does anybody has an idea what configuration or other parameters could be the cause?

* Are there yarn specific log files regarding the scheduling behavior giving a clue to what happens?

Would be great if somebody an idea or a simliar problem.

Cheers,

Matthias