Support Questions
Find answers, ask questions, and share your expertise

Who agreed with this solution

Re: Spark/Scala Error: value toDF is not a member of org.apache.spark.rdd.RDD

Ok, I finally fixed the issue. 2 things needed to be done:

 

1- Import implicits:

      Note that this should be done only after an instance of org.apache.spark.sql.SQLContext is created. It should be written as:

      val sqlContext= new org.apache.spark.sql.SQLContext(sc)
      import sqlContext.implicits._

 

 

2- Move case class outside of the method:

      case class, by use of which you define the schema of the DataFrame, should be defined outside of the method needing it. You can read more about it here:

      https://issues.scala-lang.org/browse/SI-6649

 

Cheers.

View solution in original post

Who agreed with this solution