Thanks for this clarification. I also had the same qurery ragrding memory issue while loading data. Here you cleared doubt about file loading from HDFS.  I have a similar question but the source is a local server or Cloud storage where the data size is more than driver memory ( let's say 1 GB in this case where the driver memory is 250 MB).  If I fire command val file_rdd = sc.textFile("/path or local or S3")   shoud Spark load the data or as you mentioned above will throgh exception?   Also, is there a way to print driver available memroy in Terminal?   Many Thanks, Siddharth Saraf ... View more