I am facing an issue while starting Spark History Server.
Seeing the below message in the stdout log
Fri Apr 24 16:01:49 AEST 2015 JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera Using /var/run/cloudera-scm-agent/process/581-spark-SPARK_HISTORY_SERVER as conf dir Using scripts/control.sh as process script Fri Apr 24 16:01:49 AEST 2015: Detected CDH_VERSION of  Fri Apr 24 16:01:49 AEST 2015: Starting Spark History Server Running [/opt/cloudera/parcels/CDH-5.2.4-1.cdh5.2.4.p0.3/lib/spark/bin/spark-class org.apache.spark.deploy.history.HistoryServer -d Error: Cannot find configuration directory: /var/run/cloudera-scm-agent/process/581-spark-SPARK_HISTORY_SERVER/hadoop-conf/user/spark/applicationHistory]
Does anyone know that the issue could be?
/var/run/cloudera-scm-agent/process is a tmpfs mount point that Cloudera Manager deploys the service configuration files to. The fact that the configuation directory is missing suggests that it has either been deleted or the tmpfs filesystem has been unmounted. Try restarting the Cloudera Manager agent on the node that is running the Spark History Server and then attempt to restart the service.