question Re: Diagnostics: Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143. in Support Questions

Diagnostics: Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143.

shindo_yasuhiro — Thu, 23 May 2019 17:00:55 GMT

Yarn Application status was failed, and displayed follows.

Diagnostics: Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143.

And Yarn Log Entries includes following message.

ERROR CoarseGrainedExecutorBackend: Executor self-exiting due to : Driver XX.XX.XX.XX:46869 disassociated! Shutting down.
INFO DiskBlockManager: Shutdown hook called

Could you tell me the checkpoint to solve this problem?

Best Regards.

Re: Diagnostics: Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143.

Shu_ashu — Fri, 24 May 2019 11:36:24 GMT

@Yasuhiro Shindo

Container killed exit code most of the time is due to memory overhead

If you haven't specified spark.yarn.driver.memoryOverhead or spark.yarn.executor.memoryOverhead these params in your spark submit then add these params (or) if you have specified then increase the already configured value.

Please refer to this link to decide overhead value.

Re: Diagnostics: Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143.

shindo_yasuhiro — Fri, 24 May 2019 15:11:56 GMT

Shu-san

Thank you for your early response.
I will feedback to my team members about this matter.

I'll let you know if I have any questions.

Best Regards.

Re: Diagnostics: Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143.

AKR — Thu, 05 Sep 2019 15:34:04 GMT

Hi,

Exit code 143 is related to Memory/GC issues. Your default Mapper/reducer memory setting may not be sufficient to run the large data set. Thus, try setting up higher AM, MAP and REDUCER memory when a large yarn job is invoked.

For more please refer to this link.

https://stackoverflow.com/questions/42972908/container-killed-by-the-applicationmaster-exit-code-is-143

Thanks

AKR