Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

How to process list of RDDs (List[RDD]) (foreach Alternative)

Contributor

How to process list of RDDs, foreach is sequential is there an alternative?

val param = RDD[(id, testNo, catId, value)] 

val key = param.map(f => (f.catId, f.testNo)).distinct.collect.toList.par

key.foreach( key => { 
// kmeans processing 
// get standard deviation 
// computation 
// etc... etc.. 
})
1 ACCEPTED SOLUTION

Contributor

I use Futures and Await. Please see stackoverflow answer.

View solution in original post

1 REPLY 1

Contributor

I use Futures and Await. Please see stackoverflow answer.