Skip to main content
loridigia
Gold ⭐️
January 26, 2022
Question

Yarn: Required AM memory(5273+527) is above the max threshold (3072 MB)

Hello everyone, i have a problem i set from Ambari those values:

yarn.nodemanager.resource.memory-mb = 7820

yarn.scheduler.minimum-allocation-mb = 1024
yarn.scheduler.maximum-allocation-mb = 7820

I restart Yarn but the same error keeps coming, why i can't have a bigger AM memory than 3072? Where this 3072 MB comes from?? Should i edit some other settings, cause i cant found any?

Thanks

10 replies

Amithsha
Gold ⭐️
Gold ⭐️
January 27, 2022

AM Memory is from this property  yarn.app.mapreduce.am.resource.mb 

You can set the am memory and tunning the value of the above property.

 

Make sure to mark the answer as the accepted solution. If it resolves your issue!

Translate
loridigia
loridigiaAuthor
Gold ⭐️
January 27, 2022

I'm not using MapReduce actually, i'm using Spark, so i'm submitting via Spark-Submit.
In fact MapReduce yarn.app.mapreduce.am.resource.mb is at 1 GB but the error says 3072.

Is it possibile that modifing the Yarn values from Ambari doesn't actually update those values? Even after reboot? 

Translate
Amithsha
Gold ⭐️
Gold ⭐️
January 27, 2022

Can you try passing this spark config on your spark shell /spark submit 

spark.yarn.am.memory=1g

Make sure to mark the answer as the accepted solution. If it resolves your issue ! 

Translate