Is
there a way to set minimum/maximum number of containers for an application?
What I am observing in my cluster is that, when YARN try to submit a job. It
pretty much put all the available resources to a given job depending on
the queue setting. When the next jobs comes in to the queue let’s say after 5
sec, it tries to find out how much resource are available. At this point of
time, when all the resources are given to the 1 st job which is still
running, it allots minimum allocation set-up on the cluster. This create a
large gap between 2 jobs, mean the same work load was submitted to the cluster.
The 1 st jobs completed may be in 10 minutes because it got lot of
resource. But the 2 nd job when it came in got minimum allocation
took 3 hours to complete. I am trying to avoid such big gap of execution time.
... View more
