the server node have 32gb ram, and he only accept spark submit jobs (he does not act as client\worker) each worker node is one of two servers types: 16 core 64GB or 48 core 196GB and the workers nodes have only Metrics Monitor / NodeManager installed all the configuration is on default. when running large job i don't mind the minute hold up, but when running short job should be over under 1 minute (for example 500 jobs (each take 30 seconds on one core) should be over under 1 minute when have enough cpu\ram to allocate, i think that the problem is the delay of actual job starting time (i can see the process start on by running top on command line on shell on the worker) 30-60 seconds after the submit is received., i see some java tasks manly regarding the creation on the container
... View more
