Support Questions
Find answers, ask questions, and share your expertise

Recommended configs for enabling G1GC in NameNode

Recommended configs for enabling G1GC in NameNode

Explorer

Anyone has a good experience for enabling G1GC in NameNode?

We need to increase the heap size of our NameNode to be larger than 120GB. CMS may not be suitable for such a large heap so we are considering using G1GC.

There're few documents about preference settings. I found Hortonworks has a relative tech blog but is still in "Technical Preview" state: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_hdfs-administration/content/ch_g1gc_garb...

There's a suggestion that I'm quite confused:

  • For large clusters (>50M files), MaxGCPauseMillis should be set to 4000.

Are there any rationale for this? Why is it acceptable to have a pause up to 4s? The pause delay will be added into the latency of upper systems like HBase. Why shouldn't we set it to a lower value like 400?