Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Yarn: Required AM memory(5273+527) is above the max threshold (3072 MB)
Labels:
- Labels:
-
Apache Ambari
-
Apache Spark
-
Apache YARN
Contributor
Created 01-26-2022 04:32 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello everyone, i have a problem i set from Ambari those values:
yarn.nodemanager.resource.memory-mb = 7820
yarn.scheduler.minimum-allocation-mb = 1024
yarn.scheduler.maximum-allocation-mb = 7820
I restart Yarn but the same error keeps coming, why i can't have a bigger AM memory than 3072? Where this 3072 MB comes from?? Should i edit some other settings, cause i cant found any?
Thanks
10 REPLIES 10
Contributor
Created 02-12-2022 10:42 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
INFO: Exception in thread "main" java.lang.IllegalArgumentException: Required AM memory
Above error is for AM and not for executors, hence you need to set the AM memory as
| spark.yarn.am.memory=2g |
- « Previous
-
- 1
- 2
- Next »