cdh集群yarn可用内存180g，用户提交spark任务基于yarn提交只能使用120g

[#1]

[#2]

Welcome to the community @mrTao. 

According to google translate your issue is around "CDH cluster yarn can use 180g of available memory, and users can only use 120g to submit spark tasks based on yarn submission". The image also indicated that YARN is using 72 cores and 174GiB of memory. 

Perhaps @Bharati or @rrrajesh will have some input. 

[#3]

Hi @mrTao 

It is not good idea to use entire memory Cluster YARN memory.  You can tune the memory from spark side by adjusting the memory using following parameters:

--conf spark.executor.instances=5
--conf spark.driver.memory=10g
--conf spark.driver.memoryOverhead=1g
--conf spark.executor.memory=10g
--conf spark.executor.memoryOverhead=1g

With the above memory configuration, YARN will allocate 66gb(Executor memory 11gb * Executor Instances(5) + Driver memory 11gb = 55g + 11g = 66g).

Better check your spark-submit once again and tune the above parameters according to your requirement.