Support Questions

Find answers, ask questions, and share your expertise

spark yarn-client mode failed due to OutOfMemoryError Exception

avatar
New Member

Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "dispatcher-event-loop-0"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "IPC Client (724929395) connection to /172.22.103.121:39927 from job_1471597554341_0133"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "main"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "LeaseRenewer:hrt_qa@nat-r7-movs-oozie-1-5.openstacklocal:8020"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "shuffle-server-0"

End of LogType:stderr

1 ACCEPTED SOLUTION

avatar
New Member

You can try to give --driver-memory to 2g in spark-submit command and see if it helps

View solution in original post

2 REPLIES 2

avatar
Rising Star

You can probably increase AM memory in "spark.yarn.am.memory", see if it helps. Default is 512m, start with 2g or 4g.

avatar
New Member

You can try to give --driver-memory to 2g in spark-submit command and see if it helps