- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Weird error while converting RDD[CaseClass] to DataFrames
- Labels:
-
Apache Spark
Created ‎03-17-2016 01:06 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi guys, I a csv which contains the contact details (name,email) like:
abc,abc@xyz.com xyz,xyz@abc.com
I am building a case class and then trying to register the RDD[CaseClass] as data frame following the steps. But endup getting error.
error: value toDF is not a member of org.apache.spark.rdd.RDD[Contact]
case class Contact(name:String,email:String) val texts = sc.textFile("\pathto\contacts.csv") // doesn't contain headers. val contacts = texts.map(s =>s.split(",")).map(s=>Contact(s(0),s(1))) val contactsDF = contacts.toDF()
Can anyone help me understand what's going wrong here?
Thanks
Created ‎03-17-2016 01:29 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well found the solution 🙂 posting answer in case if others face this issue in future.
following line was missing:
import sqlContext.implicits._
Created ‎03-17-2016 01:29 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well found the solution 🙂 posting answer in case if others face this issue in future.
following line was missing:
import sqlContext.implicits._
