Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Weird error while converting RDD[CaseClass] to DataFrames

Solved Go to solution

Weird error while converting RDD[CaseClass] to DataFrames

Expert Contributor

Hi guys, I a csv which contains the contact details (name,email) like:

abc,abc@xyz.com
xyz,xyz@abc.com

I am building a case class and then trying to register the RDD[CaseClass] as data frame following the steps. But endup getting error.

error: value toDF is not a member of org.apache.spark.rdd.RDD[Contact]

case class Contact(name:String,email:String)
val texts = sc.textFile("\pathto\contacts.csv") // doesn't contain headers.
val contacts = texts.map(s =>s.split(",")).map(s=>Contact(s(0),s(1)))
val contactsDF = contacts.toDF()

Can anyone help me understand what's going wrong here?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Weird error while converting RDD[CaseClass] to DataFrames

Expert Contributor

Well found the solution :) posting answer in case if others face this issue in future.

following line was missing:

import sqlContext.implicits._

1 REPLY 1

Re: Weird error while converting RDD[CaseClass] to DataFrames

Expert Contributor

Well found the solution :) posting answer in case if others face this issue in future.

following line was missing:

import sqlContext.implicits._

Don't have an account?
Coming from Hortonworks? Activate your account here