Weird error while converting RDD[CaseClass] to DataFrames

smartninja723 — Thu, 17 Mar 2016 20:06:29 GMT

Hi guys, I a csv which contains the contact details (name,email) like:

abc,abc@xyz.com
xyz,xyz@abc.com

I am building a case class and then trying to register the RDD[CaseClass] as data frame following the steps. But endup getting error.

error: value toDF is not a member of org.apache.spark.rdd.RDD[Contact]

case class Contact(name:String,email:String)
val texts = sc.textFile("\pathto\contacts.csv") // doesn't contain headers.
val contacts = texts.map(s =>s.split(",")).map(s=>Contact(s(0),s(1)))
val contactsDF = contacts.toDF()

Can anyone help me understand what's going wrong here?

Thanks

Re: Weird error while converting RDD[CaseClass] to DataFrames

smartninja723 — Thu, 17 Mar 2016 20:29:52 GMT

Well found the solution 🙂 posting answer in case if others face this issue in future.

following line was missing:

import sqlContext.implicits._

question Re: Weird error while converting RDD[CaseClass] to DataFrames in Archives of Support Questions (Read Only)

Weird error while converting RDD[CaseClass] to DataFrames

Re: Weird error while converting RDD[CaseClass] to DataFrames