Support Questions
Find answers, ask questions, and share your expertise

Yarn not allocating all reserved memory

Expert Contributor

Hello community!

I've a cluster with 10 datanodes, and 128 GB of RAM each.

Our devs use all RAM asigned to yarn, around 100GB per node.

But when I check grafana to check real memory consumption I found that when YARN is using the 100% of cluster resources, the memory allocated in servers nearly reach the 70% as avg.

YARN nodemanagers used memory:

14888-grafana-yarn-nodemanagers-google-chrome.jpg

System servers memory used:

14889-grafana-system-servers-google-chrome-2.png

In our development servers the breach is greater

YARN nodemanagers used memory:

14911-dev-grafana-yarn-nodemanagers-google-chrome-2.png

System servers memory used:

14912-dev-grafana-system-servers-google-chrome-3.png

Do you know any way to improve the cluster utilization, or depends on the applications executed in yarn?

Thank you in advance!

1 REPLY 1