Created 01-19-2021 11:07 AM
we have Hadoop cluster with 2 name-nodes ( active standby ) and 12 data nodes
all 12 data-nodes machines have disks for HDFS
we are before the action of `hadoop namenode -recover` , and that because we suspect about corrupted files as fsimage_0000000000001253918 or edits_0000000000001203337-0000000000001214475 etc
so to recover the hdfs meta data we can do the following
$ hadoop namenode -recover
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
21/01/19 17:56:35 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: user = hdfs
STARTUP_MSG: host = master1.sys67.com/17.2.12.78
STARTUP_MSG: args = [-recover]
STARTUP_MSG: version = 2.7.3.2.6.5.0-292
21/01/19 17:56:35 INFO namenode.NameNode: createNameNode [-recover]
You have selected Metadata Recovery mode. This mode is intended to recover lost metadata on a corrupt filesystem. Metadata recovery mode often permanently deletes data from your HDFS filesystem. Please back up your edit log and fsimage before trying this!
Are you ready to proceed? (Y/N)
(Y or N) y
the question is:
dose this action could also affected the data itself on the data-nodes machines ?
or only the meta data on namenode machines?
Created 01-25-2021 02:30 AM
@mike_bronson7 Looks like your question is covered here: https://community.cloudera.com/t5/Support-Questions/hadoop-cluster-with-active-standby-namenode-gap-...
Created 01-28-2021 12:04 AM
@mike_bronson7
Adding to @GangWar :-
To your question - dose this action could also affected the data itself on the data-nodes machines ?
No it doesnt affect data on datanode directly. This is metadata operation on namenode which when need to be run when NameNode fails to progress through the edits or fsimage then the NameNode may need to be started with -recover option.
Since the metadata has reference to the blocks on datanode, hence this is a critical operation and may incur data loss.