Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark2 service in the existing cluster

Highlighted

Spark2 service in the existing cluster

Expert Contributor

In my cluster I have spark1.6 and I am trying to add spark2. I was able to add the service but spark2 service does not come up..

here is an extract from the logs :

Not sure why its checking at spark1.6 and complaining its already up and running

resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdp/current/spark2-historyserver/sbin/start-history-server.sh' returned 1. org.apache.spark.deploy.history.HistoryServer running as process 39259.  Stop it first
1 REPLY 1

Re: Spark2 service in the existing cluster

Contributor

Hi Kumar,

It could be the case that you have a stale SHS process, so work on stopping any SHS process as PID 39259 and/or remove "rm" the SHS process id file usually under /var/run/spark2, and the restart it again.

Don't have an account?
Coming from Hortonworks? Activate your account here