Reply
Explorer
Posts: 20
Registered: ‎10-04-2017

Spark2 History server going down with Out of memory

[ Edited ]

Spark2 History server going down with Out of memory.

increased the heap size to 1gb and restarted again same issue. 

Appreacite if someone can advice?

 

part of log:

'[' -n '-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/spark2_on_yarn_spar1957cecc-SPARK2_YARN_HISTORY_SERVER-e8531d5dc1f494cbe596792e_pid{{PID}}.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh' ']'
++ replace_pid -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/spark2_on_yarn_spar1957cecc-SPARK2_YARN_HISTORY_SERVER-e8531d5dc1f494cbe596792e_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
++ echo -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/spark2_on_yarn_spar1957cecc-SPARK2_YARN_HISTORY_SERVER-e8531d5dc1f494cbe596792e_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
+

+ exec /opt/cloudera/parcels/SPARK2-2.0.0.cloudera1-1.cdh5.7.0.p0.113931/lib/spark2/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/10402-spark2_on_yarn-SPARK2_YARN_HISTORY_SERVER/spark2-conf/spark-history-server.conf
+ grep -q OnOutOfMemoryError /proc/12752/cmdline

 

Attached full log. 

Announcements