Support Questions
Find answers, ask questions, and share your expertise

Would data spill on disk if there is single partition with 10 GB of data and there is only one executor with 4 gb memory ?

What will happen for the below case?

data size ( partition size ) : 10GB

executor memory : 4 gb

number of executors : 1

persist() : MEMORY_AND_DISK

 

I assume this will throw out of memory , is my assumption correct ?