question Re: PySpark - Error initializing SparkContext in Support Questions

question Re: PySpark - Error initializing SparkContext in Support Questions https://community.cloudera.com/t5/Support-Questions/PySpark-Error-initializing-SparkContext/m-p/51888#M23513 The problem seems to be with configuration rather than dependency, I am not sure what configuration is missing.  Here is my configuration  : spark-defaults.conf : spark.authenticate=false spark.dynamicAllocation.enabled=true spark.dynamicAllocation.executorIdleTimeout=60 spark.dynamicAllocation.minExecutors=0 spark.dynamicAllocation.schedulerBacklogTimeout=1 spark.eventLog.dir=hdfs://dtest.turn.com:8020/user/spark/applicationHistory spark.eventLog.enabled=true spark.serializer=org.apache.spark.serializer.KryoSerializer spark.shuffle.service.enabled=true spark.shuffle.service.port=7337 spark.master=yarn spark.yarn.jars=hdfs://dtest.turn.com:8020/user/spark/spark-2.1-bin-hadoop/* spark.yarn.historyServer.address=<A href="http://dtest.turn.com:18088" target="_blank">http://dtest.turn.com:18088</A> spark.driver.extraLibraryPath=/opt/cloudera/parcels/CDH-5.5.4-1.cdh5.5.4.p0.9/lib/hadoop/lib/native spark.executor.extraLibraryPath=/opt/cloudera/parcels/CDH-5.5.4-1.cdh5.5.4.p0.9/lib/hadoop/lib/native spark.yarn.am.extraLibraryPath=/opt/cloudera/parcels/CDH-5.5.4-1.cdh5.5.4.p0.9/lib/hadoop/lib/native spark.yarn.config.gatewayPath=/opt/cloudera/parcels spark.yarn.config.replacementPath={{HADOOP_COMMON_HOME}}/../../..     Tue, 07 Mar 2017 18:54:31 GMT Murthy 2017-03-07T18:54:31Z