Created 03-07-2017 04:51 PM
I am trying to run some spark streaming examples online. But even before I start, I'm getting this error
Only one SparkContext may be running in this JVM (see SPARK-2243). To ignore this error, set spark.driver.allowMultipleContexts = true. The currently running SparkContext was created at: org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
I tried this below but doesn't help.
conf.set("spark.driver.allowMultipleContexts","true");
Sample code I was trying to run in HDP 2.5
import org.apache.spark._ import org.apache.spark.streaming._ val conf = new SparkConf().setAppName(appName).setMaster(master) val ssc = new StreamingContext(conf, Seconds(1))
Created 03-07-2017 05:54 PM
Hi @Adnan Alvee,
Are you using spark-shell? if yes the spark context is already generated for you (as 'sc') and you don't need to create a new one, you should be able to directly go with:
val ssc = new StreamingContext(sc, Seconds(1))
Created 03-07-2017 05:54 PM
Hi @Adnan Alvee,
Are you using spark-shell? if yes the spark context is already generated for you (as 'sc') and you don't need to create a new one, you should be able to directly go with:
val ssc = new StreamingContext(sc, Seconds(1))
Created 03-07-2017 06:25 PM
oh! that worked. Thanks a lot!