Support Questions

Find answers, ask questions, and share your expertise

Is it possible to repartition an existing RDD which is already partioned ?

 
3 REPLIES 3

@Sudharsan Ganeshkumar

Yes, you can repartition the rdd which is already partitioned. Just use .repartition

@Sandeep Nemuri Ya, i m able to use repartition for a different RDD. Is it possible to repartition the same RDD ?

@Sudharsan Ganeshkumar

RDD are immutable so when you repartition or coalesce it always creates a new RDD. When possible it's preferred to use coalesce since it could avoid shuffling (which is always expensive task). You can read more here:

https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-rdd-partitions.html

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.