Apache SPARK - Overwrite data file

[#1]

Hi experts, How can I overwrite an existing file by a new one (data update). Imagine that I've this:

1. result.map(pair => pair.swap).sortByKey(true).saveAsTextFile("FILE/results")

And Imagine that I want to do this:

1. test.map(pair => pair.swap).sortByKey(false).saveAsTextFile("FILE/results")

How can I overwrite the results of the var result to the results of the val test in same directory?

[#2 · Accepted solution]

RDD's saveAsTextFile does not give us the opportunity to do that (DataFrame's have "save modes" for things like append/overwrite/ignore). You'll have to control this prior before (maybe delete or rename existing data) or afterwards (write the RDD as a diff dir and then swap it out).