Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark history server has some issue.

avatar
Explorer

I can't see Spark history server's user interface, but only
"""
HISTORY SERVER title and 
Event log directory: hdfs://server/user/spark/history
"""

On the web UI.

Recently when I met this issue I managed to resolve it, I increased DAEMON_MEMORY_SIZE and HEAP_SIZE for the  history server. 
But it only helped for a couple of days, I have to constantly restart it.
Could someone help, please?

1 REPLY 1

avatar
Expert Contributor

Hello @monorels ,

 

Thank you for posting the query.

 

Check the total size of the files present on the event logging directory. possibly if you have huge sized files you would need to increase the spark history server's heap memory. Also check from the logs (spark history server logs) and see if you are getting any errors while spark history server replays the event logs from HDFS path. If there are no errors observed try to enable DEBUG level logging. Even still no errors logged, possibly you might need to check the number of event logs and the allocated heap memory usage.

Thanks,
Satz