question Spark History UI crashing often? in Archives of Support Questions (Read Only)

Spark History UI crashing often?

lickteig_joshua — Sat, 30 Jan 2016 05:02:11 GMT

We have experienced that the Spark History UI often requires being restarted as we find the service on port 18080 is down, returns Server 500 error. Any reason this might be the case?

HDP 2.2.4.2, Spark 1.2

It was suggested to increase the YARN heap to 8G, which fostered no improvement.

Thanks!

Joshua

Re: Spark History UI crashing often?

aervits — Sat, 30 Jan 2016 06:10:20 GMT

@Joshua Lickteig are you experiencing This ?

Re: Spark History UI crashing often?

azeltov — Tue, 02 Feb 2016 01:22:47 GMT

@Joshua can you increase the SPARK_DAEMON_MEMORY to 4g

SPARK_DAEMON_MEMORY Memory to allocate to the history server (default: 1g).

http://spark.apache.org/docs/latest/monitoring.html

Re: Spark History UI crashing often?

AKR — Sun, 08 Sep 2019 13:58:16 GMT

Hi,

Can you share the error message from spark history server logs when the spark history UI is crashing?

Thanks

AKR