Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Namenode Best Pratices / Performance tunning checklist

Solved Go to solution
Highlighted

Namenode Best Pratices / Performance tunning checklist

Does anyone have a best practices guide / performance checklist for namenode?

I know when we increase namenode_heapsize, we should also adjust namenode_opt_newsize, namenode_opt_maxnewsize, namenode_opt_permsize, namenode_opt_maxpermsize, but what are the reference calculations for this adjustment?

Also there is: dfs.namenode.handler.count and dfs.namenode.service.handler.count, as discussed in question below, but same here, what are reference values for these properties? Is it based number of nodes only or should also be increased based on usage patterns? https://community.hortonworks.com/questions/4799/is-there-any-suggested-ratio-for-dividing-the-numb....

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Namenode Best Pratices / Performance tunning checklist

New Contributor

-XX:NewSize and - XX:MaxnewSize should be 1/8 of the maximum heap size (-Xmx). So if -Xmx is set to 8GB, the --XX:NewSize=- XX:MaxnewSize should be set to 1GB.

The value of dfs.namenode.handler.count is calculated based on the number of datanodes in the cluster. @Arpit Agarwal suggested that the value for this should be ln(number of datanodes)*20. For example in a 450 node cluster it can be set to around 180.

Thanks,

Koelli

View solution in original post

3 REPLIES 3
Highlighted

Re: Namenode Best Pratices / Performance tunning checklist

New Contributor
Highlighted

Re: Namenode Best Pratices / Performance tunning checklist

New Contributor

-XX:NewSize and - XX:MaxnewSize should be 1/8 of the maximum heap size (-Xmx). So if -Xmx is set to 8GB, the --XX:NewSize=- XX:MaxnewSize should be set to 1GB.

The value of dfs.namenode.handler.count is calculated based on the number of datanodes in the cluster. @Arpit Agarwal suggested that the value for this should be ln(number of datanodes)*20. For example in a 450 node cluster it can be set to around 180.

Thanks,

Koelli

View solution in original post

Highlighted

Re: Namenode Best Pratices / Performance tunning checklist

Thank you @kmungee.

Would you have other configuration to check on namenode service?

Don't have an account?
Coming from Hortonworks? Activate your account here