Reply
Expert Contributor
Posts: 87
Registered: ‎09-17-2014

Change default StorageLevel for cache() and persist() operations

hi experts!

 

there are few storage levels which could be used for Spark persist and cache operations.

(https://umbertogriffo.gitbooks.io/apache-spark-best-practices-and-tuning/content/which_storage_level...)

 

by default MEMORY_ONLY used.

according my observation, MEMORY_AND_DISK_SER maybe more efficient for more cases for me.

i'd like to change default StorageLevel for this.

 

is someone have any idea how to do this?

 

thanks!

Announcements