Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Is it possible to repartition an existing RDD which is already partioned ?

 
3 REPLIES 3

@Sudharsan Ganeshkumar

Yes, you can repartition the rdd which is already partitioned. Just use .repartition

@Sandeep Nemuri Ya, i m able to use repartition for a different RDD. Is it possible to repartition the same RDD ?

@Sudharsan Ganeshkumar

RDD are immutable so when you repartition or coalesce it always creates a new RDD. When possible it's preferred to use coalesce since it could avoid shuffling (which is always expensive task). You can read more here:

https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-rdd-partitions.html

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.