In spark, why foreach is designed as an action?

[#1]

This is a newbie question. I understand that spark actions include count(), first(), take(), etc, each triggers execution with some sort of "output", either to console, or to storage. What I don't understand is why foreach is designed as an action?

[#2]

The foreach action in Spark is designed like a forced map (so the "map" action occurs on the executors). Foreach is useful for a couple of operations in Spark. They are required to be used when you want to guarantee an accumulator's value to be correct. In addition, they can be used when you want to move data to an external system, like a database, though typically a foreachPartition is used for that operation.

[#3]

To add to Joe's response, one more thing to note here is that foreach() action doesn't return anything. So if you want to invoke any transformations on your RDDs on executors without returning anything back to driver, you can invoke them using foreach() action which is not normally possible with other actions apart from saveAsxxx() actions.

-Prav

[#4]

You can use RDD.toLocalIterator() to bring the data to the driver (one RDD partition at a time for your low banner ad design cost😞

val array = sc.parallelize(List(1, 2, 3, 4))
for(rec <- array.toLocalIterator) { println(rec) }