question not able to import import spark.implicits._ in Archives of Support Questions (Read Only)

not able to import import spark.implicits._

amol_08 — Thu, 11 Oct 2018 12:00:57 GMT

I am using spark shell , my os is centos6 when i am trying to import spark.implicits._ , i get below error:

<console>:30: error: not found: value spark
         import spark.implicits._
                ^

Re: not able to import import spark.implicits._

amol_08 — Thu, 11 Oct 2018 14:15:09 GMT

also I get below error too :

scala> val spark = SparkSession.builder().enableHiveSupport().getOrCreate()
<console>:30: error: not found: value SparkSession
         val spark = SparkSession.builder().enableHiveSupport().getOrCreate(

Re: not able to import import spark.implicits._

amol_08 — Tue, 16 Oct 2018 18:27:10 GMT

I am able to import import spark.implicits._ , earlier i was using spark1 but launching spark2 solved the problem.

Re: not able to import import spark.implicits._

RangaReddy — Tue, 14 Jul 2020 04:41:27 GMT

Hi,

There is no package called spark.implicits.

Spark 1.x:

If you are using spark1.x version you will create sqlContext. By using sqlContext you can call sqlContext.implicits

Example:

val sc = new SparkContext(conf)
val sqlContext = new SQLContext(sc)
sqlContext.implicits

Spark 2.x:

If you are using spark2.x version you will create session object. By using session you can call spark.implicits.

Example:

val spark: SparkSession = SparkSession.builder.appName(appName).config("spark.master", "local[*]").getOrCreate
spark.implicits

Note: If you are created session object using different name then you need to call with that reference name.

For example,

val rangaSpark: SparkSession = SparkSession.builder.appName(appName).config("spark.master", "local[*]").getOrCreate
rangaSpark.implicits