Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark yarn-client mode failed due to OutOfMemoryError Exception

Solved Go to solution

spark yarn-client mode failed due to OutOfMemoryError Exception

New Contributor

Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "dispatcher-event-loop-0"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "IPC Client (724929395) connection to /172.22.103.121:39927 from job_1471597554341_0133"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "main"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "LeaseRenewer:hrt_qa@nat-r7-movs-oozie-1-5.openstacklocal:8020"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "shuffle-server-0"

End of LogType:stderr

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: spark yarn-client mode failed due to OutOfMemoryError Exception

New Contributor

You can try to give --driver-memory to 2g in spark-submit command and see if it helps

2 REPLIES 2

Re: spark yarn-client mode failed due to OutOfMemoryError Exception

Contributor

You can probably increase AM memory in "spark.yarn.am.memory", see if it helps. Default is 512m, start with 2g or 4g.

Highlighted

Re: spark yarn-client mode failed due to OutOfMemoryError Exception

New Contributor

You can try to give --driver-memory to 2g in spark-submit command and see if it helps

Don't have an account?
Coming from Hortonworks? Activate your account here