Support Questions
Find answers, ask questions, and share your expertise

spark submit take time untill the process starts


for the past few month we used the ambari with few servers.

I am trying to improve the time it take from submiting the job, to the time the jobs actualy starts\

In the current setup we have 4 computers with total of 64 executors

when the job is accepted in the server it take aprox 35 seconds to initialize all exectutors and start our process, where i can find the reason and is it posibble to improve\optimize the flow ?

can i keep the executor alive ? most of them time (99.99 % of the cases ) it the cpu/ram is not changed upon submited job


we based on HDP-