https://community.cloudera.com/t5/Support-Questions/Spark-History-Server-fails-to-start-with-new-installation/m-p/312823#M225282 <P>Permisson seems to be okay</P><P>&nbsp;</P><P>[root@cdp1pub ~]# hdfs dfs -ls /user/<BR />Found 4 items<BR />drwx------ - hdfs supergroup 0 2021-03-02 00:42 /user/hdfs<BR />drwxrwx--- - mapred supergroup 0 2021-03-01 09:10 /user/history<BR />drwxr-x--x - spark spark 0 2021-03-10 05:26 /user/spark<BR />drwxr-xr-x - yarn supergroup 0 2021-03-02 00:37 /user/yarn<BR />[root@cdp1pub ~]# hdfs dfs -ls /user/spark<BR />Found 2 items<BR />drwxrwxrwt - spark spark 0 2021-03-10 05:26 /user/spark/applicationHistory<BR />drwxrwxrwt - spark spark 0 2021-03-10 05:26 /user/spark/driverLogs<BR />[root@cdp1pub ~]#</P><P>Tried to restart history server it still fails</P><P>###################</P><P>Logs</P><PRE>Wed Mar 10 20:57:31 EST 2021 JAVA_HOME=/usr/java/jdk1.8.0_232-cloudera Using -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/spark_on_yarn_spar40365358-SPARK_YARN_HISTORY_SERVER-42af0f75a56c8c9b8b467a684_pid7765.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh as CSD_JAVA_OPTS Using /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER as conf dir Using scripts/control.sh as process script CONF_DIR=/var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER CMF_CONF_DIR= Wed Mar 10 20:57:31 EST 2021: Running Spark CSD control script... Wed Mar 10 20:57:31 EST 2021: Detected CDH_VERSION of [7] Wed Mar 10 20:57:31 EST 2021: Starting Spark History Server Running [/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/spark/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER/spark-conf/spark-history-server.conf] Wed Mar 10 20:57:38 EST 2021 JAVA_HOME=/usr/java/jdk1.8.0_232-cloudera Using -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/spark_on_yarn_spar40365358-SPARK_YARN_HISTORY_SERVER-42af0f75a56c8c9b8b467a684_pid8080.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh as CSD_JAVA_OPTS Using /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER as conf dir Using scripts/control.sh as process script CONF_DIR=/var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER CMF_CONF_DIR= Wed Mar 10 20:57:39 EST 2021: Running Spark CSD control script... Wed Mar 10 20:57:39 EST 2021: Detected CDH_VERSION of [7] Wed Mar 10 20:57:39 EST 2021: Starting Spark History Server Running [/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/spark/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER/spark-conf/spark-history-server.conf] Wed Mar 10 20:57:48 EST 2021 JAVA_HOME=/usr/java/jdk1.8.0_232-cloudera Using -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/spark_on_yarn_spar40365358-SPARK_YARN_HISTORY_SERVER-42af0f75a56c8c9b8b467a684_pid8375.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh as CSD_JAVA_OPTS Using /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER as conf dir Using scripts/control.sh as process script CONF_DIR=/var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER CMF_CONF_DIR= Wed Mar 10 20:57:48 EST 2021: Running Spark CSD control script... Wed Mar 10 20:57:48 EST 2021: Detected CDH_VERSION of [7] Wed Mar 10 20:57:48 EST 2021: Starting Spark History Server Running [/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/spark/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER/spark-conf/spark-history-server.conf] Wed Mar 10 20:57:57 EST 2021 JAVA_HOME=/usr/java/jdk1.8.0_232-cloudera Using -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/spark_on_yarn_spar40365358-SPARK_YARN_HISTORY_SERVER-42af0f75a56c8c9b8b467a684_pid8757.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh as CSD_JAVA_OPTS Using /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER as conf dir Using scripts/control.sh as process script CONF_DIR=/var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER CMF_CONF_DIR= Wed Mar 10 20:57:57 EST 2021: Running Spark CSD control script... Wed Mar 10 20:57:57 EST 2021: Detected CDH_VERSION of [7] Wed Mar 10 20:57:57 EST 2021: Starting Spark History Server Running [/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/spark/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/1546336662-spark_on_yarn-SPARK_YARN_HISTORY_SERVER/spark-conf/spark-history-server.conf]</PRE> Thu, 11 Mar 2021 01:59:54 GMT PrernaU 2021-03-11T01:59:54Z