The RDD exists only as long as the spark driver lives. if one or more of the spark worker containers die the portions of the RDDs will be recomputed and cached. persist and cache at the RDD level are actually the same. persist has more options though: the default behavior of persist is StorageLevel.MEMORY_ONLY but you can persist at various different storage levels. http://spark.apache.org/docs/latest/programming-guide.html#rdd-persistence ... View more