Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP 2,3.4- Running jobs is not getting displayed in Spark UI

Solved Go to solution

Re: HDP 2,3.4- Running jobs is not getting displayed in Spark UI

New Contributor

Hope this will help anyone:

Add or modify below properties

Ambari > Spark > Configs > Custom spark-defaults

spark.eventLog.enabled = true

spark.history.fs.logDirectory = hdfs://<fs.defaultFS>/<directory-to-your-spark-logs>

Ambari > Spark > Configs > Advanced spark-defaults

spark.history.provider = org.apache.spark.deploy.history.FsHistoryProvider

Don't have an account?
Coming from Hortonworks? Activate your account here