Member since
12-20-2022
88
Posts
19
Kudos Received
9
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 515 | 05-08-2025 06:27 AM | |
| 551 | 04-02-2025 11:35 PM | |
| 449 | 03-23-2025 11:30 PM | |
| 614 | 03-06-2025 10:11 PM | |
| 1282 | 10-29-2024 11:53 PM |
04-23-2024
06:59 AM
If its a tez application, AM logs will show how much memory is currently allocated/consumed by the application & how much free resources available in the queue at that specific time. eg., 2024-04-22 23:27:20,636 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: Allocated: <memory:843776, vCores:206> Free: <memory:2048, vCores:306> pendingRequests: 0 delayedContainers: 205 heartbeats: 101 lastPreemptionHeartbeat: 100 2024-04-22 23:27:30,660 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: Allocated: <memory:155648, vCores:38> Free: <memory:495616, vCores:356> pendingRequests: 0 delayedContainers: 38 heartbeats: 151 lastPreemptionHeartbeat: 150 This allocation details will be logged frequently in Tez AM logs.
... View more
04-23-2024
06:18 AM
Hi @DanhH , Please accept this as a solution if it helped.
... View more
04-23-2024
06:18 AM
Hi @HadoopCommunity , If my solution. helped you Please accept it as a solution
... View more
04-22-2024
11:27 AM
Hi @yagoaparecidoti , You dont have to run the balancer. The blocks will recognize the racks and it will be done automatically.
... View more
04-22-2024
11:24 AM
Hi @JiHoone I think all your jobs that is running is allocated only the AM resources. Do one thing first set the "Configured Max AM limit" to 20%. Then set the Minimum user limit percent to 25% and run your jobs. Set user limit factor as 0.5.
... View more
04-22-2024
10:24 AM
hi @AyazHussain since the problem is old, I don't remember what was done to fix it. if the problem returns, I will test. thanks!
... View more
04-21-2024
10:02 PM
As a rule of thumb assign 80% of the node resources to YARN. Please go through this to modify the config according to your need. https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Queue_Properties
... View more
04-16-2024
11:59 PM
1 Kudo
Hi @mike_bronson7 Please check that if the RM is working fine or is it down. Please also check your zookeeper. Please check with telnet if you can connect to RMhost from other host.
... View more
02-15-2024
12:29 PM
1 Kudo
@Ivoz Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
- « Previous
- Next »