Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NameNode heapsize is bigger then it should be.

Solved Go to solution
Highlighted

NameNode heapsize is bigger then it should be.

Hi,

I am running HDP-2.3.4.0 and Ambari 2.2.0.0

The NameNode heap size is set to 24Gb where 22Gb are used. This is very strange since the total blocks number is 7,796,546.

The fsimage size is 19Gb. Here is the jmap output for namenode JVM

-bash-4.2$ /usr/java/jdk1.7.0_67/bin/jmap -histo 35350 | head

#num #instances #bytes class name

----------------------------------------------

1: 175738748 14270581184 [B

2: 193636970 13941861840 org.apache.hadoop.hdfs.server.namenode.INodeFileAttributes$SnapshotCopy

3: 193636970 10843670320 org.apache.hadoop.hdfs.server.namenode.snapshot.FileDiff

4: 15675953 2867785824 [Ljava.lang.Object;

5: 4 1761607776 [Lorg.apache.hadoop.util.LightWeightGSet$LinkedElement;

6: 7798931 748697376 org.apache.hadoop.hdfs.server.namenode.INodeFile

7: 7799784 499186176 org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoContiguous

According to the best practices, we could use about 9 or 10Gb heap size. But this is not the case.

What could impact the heap size and how could I troubleshot this?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NameNode heapsize is bigger then it should be.

Try clearing up some snapshots. You probably have a ton of deleted files retained for snapshots.

View solution in original post

4 REPLIES 4

Re: NameNode heapsize is bigger then it should be.

It is likely the process has not hit an allocation failure yet so GC has not kicked in. This is perfectly normal. If you want the heap usage to be lower then you can reduce the heap allocation.

Alternatively you can trigger GC quicker by adding something like -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly to your heap options.

However it's probably best to just follow our suggested heap configuration and let the Java runtime do the rest.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_command-line-installation/content/config...

Highlighted

Re: NameNode heapsize is bigger then it should be.

I feel this is not a case, since fsimage size is 19Gb for about 8,000,000 blocks - which is too much. I have updated the question with more info.

Highlighted

Re: NameNode heapsize is bigger then it should be.

Try clearing up some snapshots. You probably have a ton of deleted files retained for snapshots.

View solution in original post

Highlighted

Re: NameNode heapsize is bigger then it should be.

Thank you! Yes, we have a lot of snapshots.

Don't have an account?
Coming from Hortonworks? Activate your account here