reduceByKey(_ ++ _)

[#1]

rdd.map(kv => (kv._1, new Set[String]() + kv._2)) .reduceByKey(_ ++ _)

In the above code, what is (kv._1, new Set[String]() + kv._2)) and reduceByKey(_ ++ _).

I know reduceByKey(_+_), but not (_++_).. Please let me know if someone knows this..

Thanks!

[#2 · Accepted solution]

The first operation makes each value into a set containing that single value. ++ just adds collections together, combining elements of both sets. This is trying to build up a set of all values for each key. It can be written more simply as "groupByKey" really. Even this code could be more compact and efficient.