Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Weird error while converting RDD[CaseClass] to DataFrames

avatar
Expert Contributor

Hi guys, I a csv which contains the contact details (name,email) like:

abc,abc@xyz.com
xyz,xyz@abc.com

I am building a case class and then trying to register the RDD[CaseClass] as data frame following the steps. But endup getting error.

error: value toDF is not a member of org.apache.spark.rdd.RDD[Contact]

case class Contact(name:String,email:String)
val texts = sc.textFile("\pathto\contacts.csv") // doesn't contain headers.
val contacts = texts.map(s =>s.split(",")).map(s=>Contact(s(0),s(1)))
val contactsDF = contacts.toDF()

Can anyone help me understand what's going wrong here?

Thanks

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Well found the solution 🙂 posting answer in case if others face this issue in future.

following line was missing:

import sqlContext.implicits._

View solution in original post

1 REPLY 1

avatar
Expert Contributor

Well found the solution 🙂 posting answer in case if others face this issue in future.

following line was missing:

import sqlContext.implicits._