Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on 10-06-2017 09:43 AM - edited 09-16-2022 05:22 AM
Hi,
Unable to import SparkSession class that is available from Spark-sql version 2.x. Here is my gradle entry:
sparkVersion = '1.6.0-cdh5.11.0'
spark_sql = "org.apache.spark:spark-sql_2.10:$sparkVersion"
Here I assume spark-sql_2.10 is spark 2.x version, not 1.6.
When I expanded spark-sql_2.10-1.6.0-cdh5.11.0-sources.jar, i don't see SparkSession class in org.apache.spark.sql package.
Can you please clarify when I am not able to see SparkSession class or point to right repository etnry.
Appreciate your help.
Thanks,
Ravi Papisetti
Created 10-09-2017 04:23 PM
This issue got resolved after changing to spark 2.1 version. I understand that in cloudera mvn artifactory, number associated to spark-core-2.10 is scala version, 1.6.0 associated with CDH is spark version.
Created 10-09-2017 04:23 PM
This issue got resolved after changing to spark 2.1 version. I understand that in cloudera mvn artifactory, number associated to spark-core-2.10 is scala version, 1.6.0 associated with CDH is spark version.
Created 04-03-2018 05:41 AM
can you post your maven dependencies for spark?