Support Questions

Find answers, ask questions, and share your expertise

Difference between DataNode JVM Heap Memory Used and Memory committed

Cloudera Employee

When using the heat map in Ambari we have a select metric for DataNode JVM Heap Memory Used and another metric for DataNode JVM Heap Memory Committed. What is the difference and why is this difference important to us?

1 ACCEPTED SOLUTION

Guru

@Will Dailey,

The JVM heap can be preconfigured for memory boundaries – the initial heap size (defined by the –Xms option) and the maximum heap size (defined by the –Xmx option).

Used memory from the perspective of the JVM is Working set + Garbage.

The Committed memory is a measure of how much memory the JVM heap is really consuming.

If (–Xms<-Xmx) and (Used memory == Current heap size), the JVM is likely to grow its heap after a full garbage collection. However if (–Xmsor Current heap size ==-Xmx), either through heap growth or configuration, the heap cannot grow any further.

Both of these Values are important to debug datanode failure related Out of Memory, less PermGen Errors, GC errors.

Reference link : https://pubs.vmware.com/vfabric52/index.jsp?topic=/com.vmware.vfabric.em4j.1.2/em4j/conf-heap-manage...

View solution in original post

1 REPLY 1

Guru

@Will Dailey,

The JVM heap can be preconfigured for memory boundaries – the initial heap size (defined by the –Xms option) and the maximum heap size (defined by the –Xmx option).

Used memory from the perspective of the JVM is Working set + Garbage.

The Committed memory is a measure of how much memory the JVM heap is really consuming.

If (–Xms<-Xmx) and (Used memory == Current heap size), the JVM is likely to grow its heap after a full garbage collection. However if (–Xmsor Current heap size ==-Xmx), either through heap growth or configuration, the heap cannot grow any further.

Both of these Values are important to debug datanode failure related Out of Memory, less PermGen Errors, GC errors.

Reference link : https://pubs.vmware.com/vfabric52/index.jsp?topic=/com.vmware.vfabric.em4j.1.2/em4j/conf-heap-manage...

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.