spark yarn-client mode failed due to OutOfMemoryError Exception

[#1]

Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "dispatcher-event-loop-0"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "IPC Client (724929395) connection to /172.22.103.121:39927 from job_1471597554341_0133"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "main"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "LeaseRenewer:[email protected]:8020"
Halting due to Out Of Memory Error ...

Exception : java.lang.OutOfMemory Error thrown from the Uncaught Exception Handler in thread "shuffle-server-0"

End of LogType:stderr

[#2]

You can probably increase AM memory in "spark.yarn.am.memory", see if it helps. Default is 512m, start with 2g or 4g.

[#3 · Accepted solution]

You can try to give --driver-memory to 2g in spark-submit command and see if it helps