Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Difference between DataNode JVM Heap Memory Used and Memory committed

avatar
Cloudera Employee

When using the heat map in Ambari we have a select metric for DataNode JVM Heap Memory Used and another metric for DataNode JVM Heap Memory Committed. What is the difference and why is this difference important to us?

1 ACCEPTED SOLUTION

avatar
Guru

@Will Dailey,

The JVM heap can be preconfigured for memory boundaries – the initial heap size (defined by the –Xms option) and the maximum heap size (defined by the –Xmx option).

Used memory from the perspective of the JVM is Working set + Garbage.

The Committed memory is a measure of how much memory the JVM heap is really consuming.

If (–Xms<-Xmx) and (Used memory == Current heap size), the JVM is likely to grow its heap after a full garbage collection. However if (–Xmsor Current heap size ==-Xmx), either through heap growth or configuration, the heap cannot grow any further.

Both of these Values are important to debug datanode failure related Out of Memory, less PermGen Errors, GC errors.

Reference link : https://pubs.vmware.com/vfabric52/index.jsp?topic=/com.vmware.vfabric.em4j.1.2/em4j/conf-heap-manage...

View solution in original post

1 REPLY 1

avatar
Guru

@Will Dailey,

The JVM heap can be preconfigured for memory boundaries – the initial heap size (defined by the –Xms option) and the maximum heap size (defined by the –Xmx option).

Used memory from the perspective of the JVM is Working set + Garbage.

The Committed memory is a measure of how much memory the JVM heap is really consuming.

If (–Xms<-Xmx) and (Used memory == Current heap size), the JVM is likely to grow its heap after a full garbage collection. However if (–Xmsor Current heap size ==-Xmx), either through heap growth or configuration, the heap cannot grow any further.

Both of these Values are important to debug datanode failure related Out of Memory, less PermGen Errors, GC errors.

Reference link : https://pubs.vmware.com/vfabric52/index.jsp?topic=/com.vmware.vfabric.em4j.1.2/em4j/conf-heap-manage...