Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

SparkSession class not found in spark-sql-2.1

avatar
Contributor

Hi,

 

Unable to import SparkSession class that is available from Spark-sql version 2.x. Here is my gradle entry:

sparkVersion        = '1.6.0-cdh5.11.0'

    spark_sql          = "org.apache.spark:spark-sql_2.10:$sparkVersion"

 

Here I assume spark-sql_2.10 is spark 2.x version, not 1.6.

When I expanded spark-sql_2.10-1.6.0-cdh5.11.0-sources.jar, i don't see SparkSession class in org.apache.spark.sql package.

 

Can you please clarify when I am not able to see SparkSession class or point to right  repository etnry.

 

Appreciate your help.

 

Thanks,

Ravi Papisetti

 

 

1 ACCEPTED SOLUTION

avatar
Contributor

This issue got resolved after changing to spark 2.1 version. I understand that in cloudera mvn artifactory, number associated to spark-core-2.10 is scala version, 1.6.0 associated with CDH is spark version.

View solution in original post

2 REPLIES 2

avatar
Contributor

This issue got resolved after changing to spark 2.1 version. I understand that in cloudera mvn artifactory, number associated to spark-core-2.10 is scala version, 1.6.0 associated with CDH is spark version.

avatar
Frequent Visitor

can you post your maven dependencies for spark?