Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Yarn: Required AM memory(5273+527) is above the max threshold (3072 MB)

avatar
Contributor

Hello everyone, i have a problem i set from Ambari those values:

yarn.nodemanager.resource.memory-mb = 7820

yarn.scheduler.minimum-allocation-mb = 1024
yarn.scheduler.maximum-allocation-mb = 7820

I restart Yarn but the same error keeps coming, why i can't have a bigger AM memory than 3072? Where this 3072 MB comes from?? Should i edit some other settings, cause i cant found any?

Thanks

10 REPLIES 10

avatar
Contributor

INFO: Exception in thread "main" java.lang.IllegalArgumentException: Required AM memory

Above error is for AM and not for executors, hence you need to set the AM memory as

spark.yarn.am.memory=2g