Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Reinstalling spark

Reinstalling spark



I had problems with Spark and my solution was reinstalling the service but it results a bad idea, It did the following steps:


Ensuring that the expected software releases are installed on hosts. > OK

Successfully deployed all client configurations. > OK

Finished waiting > OK

Start Zookeeper > OK

Start HDFS > OK




com.cloudera.cmf.service.config.ConfigGenException: Conflicting yarn extensions provided by more than one service. Yarn extension key [spark_shuffle], First value [YarnAuxServiceExtension{className='', auxServiceId='spark_shuffle', configs={spark.shuffle.service.port=7337, spark.authenticate=true}, service=Spark}], Second value [YarnAuxServiceExtension{className='', auxServiceId='spark_shuffle', configs={spark.shuffle.service.port=7337, spark.authenticate=false}, service=Spark}].


However cloudera manager shows Spark as OK but spark-shell stop when try to initiate spark context:


Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.6.0

Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_67)
Type in expressions to have them evaluated.
Type :help for more information.


Re: Reinstalling spark


did you remove the spark service nice and clean without any issue before re-install ? 

could you provide us the logs on spark service 

Don't have an account?
Coming from Hortonworks? Activate your account here