Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

not able to import import spark.implicits._

avatar

I am using spark shell , my os is centos6 when i am trying to import spark.implicits._ , i get below error:

<console>:30: error: not found: value spark
         import spark.implicits._
                ^
1 ACCEPTED SOLUTION

avatar

I am able to import import spark.implicits._ , earlier i was using spark1 but launching spark2 solved the problem.

View solution in original post

3 REPLIES 3

avatar

also I get below error too :

scala> val spark = SparkSession.builder().enableHiveSupport().getOrCreate()
<console>:30: error: not found: value SparkSession
         val spark = SparkSession.builder().enableHiveSupport().getOrCreate(

avatar

I am able to import import spark.implicits._ , earlier i was using spark1 but launching spark2 solved the problem.

avatar
Super Collaborator

Hi,

 

There is no package called spark.implicits. 

Spark 1.x:

If you are using spark1.x version you will create sqlContext. By using sqlContext  you can call sqlContext.implicits

Example:

val sc = new SparkContext(conf)
val sqlContext = new SQLContext(sc)
sqlContext.implicits

Spark 2.x:

If you are using spark2.x version you will create session object. By using session you can call spark.implicits.

Example:

val spark: SparkSession = SparkSession.builder.appName(appName).config("spark.master", "local[*]").getOrCreate
spark.implicits

Note: If you are created session object using different name then you need to call with that reference name. 

For example,

val rangaSpark: SparkSession = SparkSession.builder.appName(appName).config("spark.master", "local[*]").getOrCreate
rangaSpark.implicits