- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
SparkSession class not found in spark-sql-2.1
- Labels:
-
Apache HBase
Created on ‎10-06-2017 09:43 AM - edited ‎09-16-2022 05:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Unable to import SparkSession class that is available from Spark-sql version 2.x. Here is my gradle entry:
sparkVersion = '1.6.0-cdh5.11.0'
spark_sql = "org.apache.spark:spark-sql_2.10:$sparkVersion"
Here I assume spark-sql_2.10 is spark 2.x version, not 1.6.
When I expanded spark-sql_2.10-1.6.0-cdh5.11.0-sources.jar, i don't see SparkSession class in org.apache.spark.sql package.
Can you please clarify when I am not able to see SparkSession class or point to right repository etnry.
Appreciate your help.
Thanks,
Ravi Papisetti
Created ‎10-09-2017 04:23 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This issue got resolved after changing to spark 2.1 version. I understand that in cloudera mvn artifactory, number associated to spark-core-2.10 is scala version, 1.6.0 associated with CDH is spark version.
Created ‎10-09-2017 04:23 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This issue got resolved after changing to spark 2.1 version. I understand that in cloudera mvn artifactory, number associated to spark-core-2.10 is scala version, 1.6.0 associated with CDH is spark version.
Created ‎04-03-2018 05:41 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
can you post your maven dependencies for spark?
