Smartsense recommendation for Namenode Metadata directory?

[#1]

Our smartsense bundle is reportig a critical Namenode metadata alert, the alert states to dedicate a disk for Namenode Metadata, which I agree with.

The recommendation is also asking to remove the NFS mount(ISILON server) for Namenode Metadata backup from the list of Namenode directories. Basically we have mounted an Isilon server for the purpose of live namenode metadata backup in case both Namenode and standby Namenode machines have catastrophic failure. Do we know if this is actually required to be removed from the list of Namenode directories and what impact can it possibly have if not removed?

[#2 · Accepted solution]

@Anil Bagga, SmartSense recommends that you do not use root file system for storing NN or DN data/metadata. There have been numerous cases where using root fs for storing data has resulted into slowness (as multiple writes happening on same disk) and in some cases crashes due to root file system being filled up and no space to write NN metadata. That's why SmartSense recommends you to avoid these problems. So you should make sure that you are setup in such a way that you don't get into either of these situations.

[#3]

Hi Sheetal, thanks for replying. I agree that we should dedicate a disk for NN metadata and remove the root FS from the Namenode metadata directories list, but my questions is regarding the NFS mount(ISILON) that we have put in place as a third copy of the Namenode metadata in case of catastrophic failure of both standby and active namenode machines?

[#4]

You can have additional copy for backup/redundancy purpose. SmartSense may not be recommending to remove it, but rather "move" it. As part of move, you would need to copy the metadata to new directory path and then update configurations (remove old directory path and add new directory path, this may be just confusion in words used.)

[#5]

Thanks for clarifying Sheetal. Appreciate your answer.