Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark2 History server going down with Out of memory

Spark2 History server going down with Out of memory

Contributor

Spark2 History server going down with Out of memory.

increased the heap size to 1gb and restarted again same issue. 

Appreacite if someone can advice?

 

part of log:

'[' -n '-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/spark2_on_yarn_spar1957cecc-SPARK2_YARN_HISTORY_SERVER-e8531d5dc1f494cbe596792e_pid{{PID}}.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh' ']'
++ replace_pid -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/spark2_on_yarn_spar1957cecc-SPARK2_YARN_HISTORY_SERVER-e8531d5dc1f494cbe596792e_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
++ echo -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/spark2_on_yarn_spar1957cecc-SPARK2_YARN_HISTORY_SERVER-e8531d5dc1f494cbe596792e_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
+

+ exec /opt/cloudera/parcels/SPARK2-2.0.0.cloudera1-1.cdh5.7.0.p0.113931/lib/spark2/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/10402-spark2_on_yarn-SPARK2_YARN_HISTORY_SERVER/spark2-conf/spark-history-server.conf
+ grep -q OnOutOfMemoryError /proc/12752/cmdline

 

Attached full log. 

1 REPLY 1
Highlighted

Re: Spark2 History server going down with Out of memory

Cloudera Employee

Hi,

 

We are unable to view the logs. Could you please share the full log for the analysis?

 

Thanks

AK