How to find number of containers in context of spark and yarn? How exactly memory per container is calculated? How to solve container memory beyond physical limits?

[#1]

@Dhana Shekar there is a description at https://community.hortonworks.com/questions/29469/yarn-container-size-flexible-to-satisfy-what-appli... that might help with a couple of your questions.

[#2]

Thanks much for timely reply. Very useful. Just found below links also useful.

https://community.hortonworks.com/articles/42803/spark-on-yarn-executor-resource-allocation-optimiz....