Created 01-21-2016 01:03 PM
During maintenance of our production cluster, one of the the datanode (of 100+ datanodes) went bad. We decided to start the cluster leaving that node.
However Namenode was stuck in safemode with below message:
Safe mode is ON. The reported blocks 8900092 needs additional 6476 blocks to reach the threshold 1.0000 of total blocks 8906567.
Is it okay to force NN out of safe mode?
What are the things to check to ensure that?
My understanding is, it is enough to check that there are no missing and corrupt files using hadoop fsck. Please confirm.
Created 01-21-2016 01:13 PM
My understanding is, it is enough to check that there are no missing and corrupt files using hadoop fsck - This is correct.
hadoop dfsadmin -safemode leave
If you are not sure then "engage support"
Created 01-21-2016 01:10 PM
Those blocks are already replicated or will be reconstructed so from the survivng nodes
In order to forcefully let the namenode leave safemode, following command should be executed depending on the version of
hadoop bin/hadoop dfsadmin -safemode leave
hdfs dfsadmin -safemode leave
Created 01-21-2016 01:16 PM
Thanks, but how to ensure that?
Created 01-21-2016 01:13 PM
My understanding is, it is enough to check that there are no missing and corrupt files using hadoop fsck - This is correct.
hadoop dfsadmin -safemode leave
If you are not sure then "engage support"
Created 01-21-2016 01:19 PM
Safe mode is ON. The reported blocks 8900092 needs additional 6476 blocks to reach the threshold 1.0000 of total blocks 8906567
See this article https://community.hortonworks.com/articles/1949/how-to-leave-hdfs-and-name-node-safemode.html