NameNode is running at 75% to 85% heap usage even when the cluster is at near idle state.
As per the following link, we should set the name node size at 1gb and Young Generation Size should be 128mb but we have set the heap size at 3gb and Young Generation Size at 384mb.
Should i change it to 1gb and 128mb as per the Hortonworks recommendation? or leave the settings as is and change some other parameter to bring the usage down? Please advise
Some other info
Total HDFS Data Size < 1 TB
Number of Files < 160k
Block Size 128 mb
The settings we currently have are recommended by Ambari itself, but the client is asking why the namenode usage is so high when there is no load on it. does it make sense to increase the heapsize to 4gb to bring the usage percentage down? or we will run into issues if we increase from 3gb to 4gb? please adivse
Usually if you have less than a million files you can go with 1024MB RAM for the Namenode (128MB -XX:NewSize -XX:MaxNewSize). You can also increase it to 3GB to be on the safe side, the Young Generation size should be 1/8 of the NN heap size.
What does your hardware setup look like? (VM or bare metal?) Whats the total memory that is available on the NN machine?
I have seen a similar heap usage on one of my test clusters. I had 1GB heap configured, about 15K blocks and the heap usage was at 25%. After restarting the Namenode the heap usage dropped to 6%.
This is not a VM and it has 256gb RAM and 64cores CPU.
We already have 3gb for Name Node heap and 384mb young generation size. So I am having tough time explaining to the customer on why the heap usage is high when there is no load. What takes the name node heap to that high when we have < 120k files and 200gb hdfs data with 128mb block size?
and also we tried to restart the name node to solve this problem. It helped to bring the heap size down but it went back to >75% after a day.
I'm facing the same problem with ambari 188.8.131.52 version.
For the moment, i increase the memory size to 3Gb. Do you get the real reason behind this? It'is a normal behaviour of recent version of ambari? thanks in advance.