Reply
Explorer
Posts: 11
Registered: ‎09-09-2015

Cloudera Navigator Metadata Server Java Heap Size

Cloudera Navigator Metadata Server Java Heap Size by default is only 2 GB. We have changed this to 20GB already and Metadata server still crashes with unexpected exits.

 

stdout.log has following entry

 

# java.lang.OutOfMemoryError: PermGen space

# -XX:OnOutOfMemoryError="/usr/lib64/cmf/service/common/killparent.sh"

#   Executing /bin/sh -c "/usr/lib64/cmf/service/common/killparent.sh"...

 

How to calculate (based on which factors) how much memory to allocate?

 

Explorer
Posts: 11
Registered: ‎09-09-2015

Re: Cloudera Navigator Metadata Server Java Heap Size

Looks like there is memory leak issue. Following warning is in the logs. Anyone has any ideas on how to resolve this?

 

A C3P0Registry mbean is already registered. This probably means that an application using c3p0 was undeployed, but not all PooledDataSources were closed prior to undeployment. This may lead to resource leaks over time. Please take care to close all PooledDataSources.

Explorer
Posts: 11
Registered: ‎09-09-2015

Re: Cloudera Navigator Metadata Server Java Heap Size

We were increasing Heap size but OOM error is related to PermGen space and not Heap space.
 

Added the option XX:MaxPermSize=2048m
 

Hopefully this will resolve. Monitoring....

Highlighted
Explorer
Posts: 22
Registered: ‎02-17-2015

Re: Cloudera Navigator Metadata Server Java Heap Size

We are facing the same problem. The service autorestarts and creates multiple headDumps on /tmp/ of ~3Gb.

Well spot that the OOM-error is occuring due to PermGen. The default value for all mgnt-services is

 

Java Configuration Options for Navigator Metadata Server:
-XX:MaxPermSize=196Mb  (default)

 

 

We have boosted it up a bit to 256 and increase the heap as well.

 

Here is a link to the Cloudera Recommendation:

Cloudera-documenations-MetadataServer-configuration

 

 

Formula = (elements + relations) * 200. You can find these in the logs:

grep NavServerUtil /pathtologs/cloudera-scm-navigator/*

 

In our case 3 Gb heap should be more than enough

elements1496854   
relations1700992   
 31978462006395692000,6
  multiplybytesgb
Announcements