Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

How to find number of containers in context of spark and yarn? How exactly memory per container is calculated? How to solve container memory beyond physical limits?

Explorer
 
2 REPLIES 2

Expert Contributor

@Dhana Shekar there is a description at https://community.hortonworks.com/questions/29469/yarn-container-size-flexible-to-satisfy-what-appli... that might help with a couple of your questions.

Explorer

Thanks much for timely reply. Very useful. Just found below links also useful.

https://community.hortonworks.com/articles/42803/spark-on-yarn-executor-resource-allocation-optimiz....