Created 04-05-2016 03:09 AM
During full restore of HDFS from local disk is name node reformat required? Since this is a full restore, the name node would point to block with do not exist. I believe reformat name node would be required but need confirmation from community.
Created 04-05-2016 06:46 PM
@Sunile Manjee Can you please clarify what you mean by "full restore" in this case? A truly full restore of HDFS would restore to a cross-cluster consistent Point-in-Time (PiT) snapshot for all datanodes AND namenode. Therefore the namenode's idea of existing blocks would match the blocks existing on the datanodes after the restore. If the PiT snapshot was not perfect (for instance, if some HDFS files were in the midst of being written during the process of taking the snapshot), then you might need to do some cleanup of the blocks for those affected files. But not via a reformat.
In general, a reformat of HDFS erases everything in it, just as with reformat of any file system. So it seems unlikely this is what you want. If you really do want a reformat, then there is no need to do a restore first; it is all going away anyway.
How was your snapshot taken, and how are you doing the full restore?
Created 04-05-2016 06:46 PM
@Sunile Manjee Can you please clarify what you mean by "full restore" in this case? A truly full restore of HDFS would restore to a cross-cluster consistent Point-in-Time (PiT) snapshot for all datanodes AND namenode. Therefore the namenode's idea of existing blocks would match the blocks existing on the datanodes after the restore. If the PiT snapshot was not perfect (for instance, if some HDFS files were in the midst of being written during the process of taking the snapshot), then you might need to do some cleanup of the blocks for those affected files. But not via a reformat.
In general, a reformat of HDFS erases everything in it, just as with reformat of any file system. So it seems unlikely this is what you want. If you really do want a reformat, then there is no need to do a restore first; it is all going away anyway.
How was your snapshot taken, and how are you doing the full restore?
Created 04-08-2016 06:59 AM
Hi @Sunile Manjee, have you seen the answers? Please consider to accept/upvote the helpful ones.