Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

RDD as val and var definitions

RDD as val and var definitions

New Contributor

RDD is considered immutable ie unchanged.Can someone explain why is RDD immutable? I tried to create an RDD with val and var like given below. I can see i was able to change RDD definitin created using var. If its immutable why was I able to use var to create an RDD?


cala> var df=sc.textFile("/user/cloudera/problem5/text")
df: org.apache.spark.rdd.RDD[String] = /user/cloudera/problem5/text MapPartitionsRDD[13] at textFile at <console>:27

scala> val db=sc.textFile("/user/cloudera/problem5/text"
| )
db: org.apache.spark.rdd.RDD[String] = /user/cloudera/problem5/text MapPartitionsRDD[15] at textFile at <console>:27

scala> df=sc.textFile("/user/cloudera/problem5/text-uncompress")
df: org.apache.spark.rdd.RDD[String] = /user/cloudera/problem5/text-uncompress MapPartitionsRDD[17] at textFile at <console>:29



Re: RDD as val and var definitions

Master Collaborator
The RDD is always immutable. It is just the definiton of the variable. In the "df" case you just assigned a new immutable RDD to a "mutable" variable call "df".