A simple answer is NO you cannot remove the SNN in a non NameNode HA setup reasons below:
Secondary NameNode(SNN) and Standby NameNode are mutually exclusive,enabling HA is not mandatory. But, when it is enabled, you can't use Secondary Namenode. So, either Secondary Namenode is enabled OR Standby Namenode is enabled.
Secondary NameNode downloads the FsImage and EditLogs from the NameNode and then it merges EditLogs with the Fsimage periodically. It keeps edits log size within a limit. After that, it stores the modified FsImage into persistent storage. So we can use FsImage in case of NameNode failure.
Secondary namenode is just a helper for Namenode.
- It gets the edit logs from the namenode in regular intervals and applies to fsimage.
- Once it has new fsimage, it copies back to namenode.
- Namenode will use this fsimage for the next restart, which will reduce the startup time.
- Secondary Namenode's whole purpose is to have a checkpoint in HDFS. Its just a helper node for namenode.
That’s why it also known also as checkpoint node,But, It can't replace namenode on namenode's failure.NameNode is single point of Failure (SPOF). If namenode fails, all clients would unable to read/write files. In such event, whole cluster will be out of service until new namenode is up.
The standby namenode provides automatic failover in case Active Namenode (can be simply called 'Namenode' if HA is not enabled) fails.You need a Zookeeper cluster (quorum of 3) to add a Standby namenode hence enabling NameNode HA To overcome this issue;
Standby NameNode comes into picture. It does three things:
- Merging fsimage and edits-log files. (Secondary-namenode's work).
- Receive online updates of the file system meta-data using journalnoodes.
- Apply the changes to its memory state and persist them on disks just like the name-node does.
Thus at any time the Backup node contains an up-to-date image of the namespace both in memory and on local disk(s). Cluster will switch over to the new name-node (this standby-node) if the active namenode dies High availability feature provides an extra NameNode to hadoop architecture,this feature provides automatic failover. If active NameNode fails, then standby-Namenode takes all the responsibility of active node. And cluster continues to work.
I have attached 2 visuals to help you understand
HTH
A simple answer is NO you cannot remove the SNN in a non NameNode HA setup reasons below:
Secondary NameNode(SNN) and Standby NameNode are mutually exclusive,enabling HA is not mandatory. But, when it is enabled, you can't use Secondary Namenode. So, either Secondary Namenode is enabled OR Standby Namenode is enabled.
Secondary NameNode downloads the FsImage and EditLogs from the NameNode and then it merges EditLogs with the Fsimage periodically. It keeps edits log size within a limit. After that, it stores the modified FsImage into persistent storage. So we can use FsImage in case of NameNode failure.
Secondary namenode is just a helper for Namenode.
- It gets the edit logs from the namenode in regular intervals and applies to fsimage.
- Once it has new fsimage, it copies back to namenode.
- Namenode will use this fsimage for the next restart, which will reduce the startup time.
- Secondary Namenode's whole purpose is to have a checkpoint in HDFS. Its just a helper node for namenode.
That’s why it also known also as checkpoint node,But, It can't replace namenode on namenode's failure.NameNode is single point of Failure (SPOF). If namenode fails, all clients would unable to read/write files. In such event, whole cluster will be out of service until new namenode is up.
The standby namenode provides automatic failover in case Active Namenode (can be simply called 'Namenode' if HA is not enabled) fails.You need a Zookeeper cluster (quorum of 3) to add a Standby namenode hence enabling NameNode HA To overcome this issue;
Standby NameNode comes into picture. It does three things:
- Merging fsimage and edits-log files. (Secondary-namenode's work).
- Receive online updates of the file system meta-data using journalnoodes.
- Apply the changes to its memory state and persist them on disks just like the name-node does.
Thus at any time the Backup node contains an up-to-date image of the namespace both in memory and on local disk(s). Cluster will switch over to the new name-node (this standby-node) if the active namenode dies High availability feature provides an extra NameNode to hadoop architecture,this feature provides automatic failover. If active NameNode fails, then standby-Namenode takes all the responsibility of active node. And cluster continues to work.
I have attached 2 visuals to help you understand
HTH
