Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Yarn not allocating all reserved memory

Highlighted

Yarn not allocating all reserved memory

Expert Contributor

Hello community!

I've a cluster with 10 datanodes, and 128 GB of RAM each.

Our devs use all RAM asigned to yarn, around 100GB per node.

But when I check grafana to check real memory consumption I found that when YARN is using the 100% of cluster resources, the memory allocated in servers nearly reach the 70% as avg.

YARN nodemanagers used memory:

14888-grafana-yarn-nodemanagers-google-chrome.jpg

System servers memory used:

14889-grafana-system-servers-google-chrome-2.png

In our development servers the breach is greater

YARN nodemanagers used memory:

14911-dev-grafana-yarn-nodemanagers-google-chrome-2.png

System servers memory used:

14912-dev-grafana-system-servers-google-chrome-3.png

Do you know any way to improve the cluster utilization, or depends on the applications executed in yarn?

Thank you in advance!

1 REPLY 1
Don't have an account?
Coming from Hortonworks? Activate your account here