Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 03-07-2017 04:51 PM
I am trying to run some spark streaming examples online. But even before I start, I'm getting this error
Only one SparkContext may be running in this JVM (see SPARK-2243). To ignore this error, set spark.driver.allowMultipleContexts = true. The currently running SparkContext was created at: org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
I tried this below but doesn't help.
conf.set("spark.driver.allowMultipleContexts","true");Sample code I was trying to run in HDP 2.5
import org.apache.spark._ import org.apache.spark.streaming._ val conf = new SparkConf().setAppName(appName).setMaster(master) val ssc = new StreamingContext(conf, Seconds(1))
Created 03-07-2017 05:54 PM
Hi @Adnan Alvee,
Are you using spark-shell? if yes the spark context is already generated for you (as 'sc') and you don't need to create a new one, you should be able to directly go with:
val ssc = new StreamingContext(sc, Seconds(1))
Created 03-07-2017 05:54 PM
Hi @Adnan Alvee,
Are you using spark-shell? if yes the spark context is already generated for you (as 'sc') and you don't need to create a new one, you should be able to directly go with:
val ssc = new StreamingContext(sc, Seconds(1))
Created 03-07-2017 06:25 PM
oh! that worked. Thanks a lot!