Reply
Rad
New Contributor
Posts: 1
Registered: ‎09-18-2018

RDD as val and var definitions

RDD is considered immutable ie unchanged.Can someone explain why is RDD immutable? I tried to create an RDD with val and var like given below. I can see i was able to change RDD definitin created using var. If its immutable why was I able to use var to create an RDD?

 

cala> var df=sc.textFile("/user/cloudera/problem5/text")
df: org.apache.spark.rdd.RDD[String] = /user/cloudera/problem5/text MapPartitionsRDD[13] at textFile at <console>:27

scala> val db=sc.textFile("/user/cloudera/problem5/text"
| )
db: org.apache.spark.rdd.RDD[String] = /user/cloudera/problem5/text MapPartitionsRDD[15] at textFile at <console>:27

scala> df=sc.textFile("/user/cloudera/problem5/text-uncompress")
df: org.apache.spark.rdd.RDD[String] = /user/cloudera/problem5/text-uncompress MapPartitionsRDD[17] at textFile at <console>:29

 

Master
Posts: 377
Registered: ‎07-01-2015

Re: RDD as val and var definitions

The RDD is always immutable. It is just the definiton of the variable. In the "df" case you just assigned a new immutable RDD to a "mutable" variable call "df".
Announcements