Support Questions
Find answers, ask questions, and share your expertise

Yarn resource waste with dynamic allocation enabled


with dynamic allocation enabled, does yarn avoid resource waste?


assume a spark job requests:

--num-executors 50

--executor-memory 64gb

--executor-cores 10


but at runtime, the job only uses 10 executors, 2gb memory and 2 cores per executor.

does the resource manager reclaim the waste?