question Re: Name Node Reformat after hdfs restore in Archives of Support Questions (Read Only)

Name Node Reformat after hdfs restore

sunile_manjee — Tue, 05 Apr 2016 10:09:59 GMT

During full restore of HDFS from local disk is name node reformat required? Since this is a full restore, the name node would point to block with do not exist. I believe reformat name node would be required but need confirmation from community.

Re: Name Node Reformat after hdfs restore

mfoley — Wed, 06 Apr 2016 01:46:14 GMT

@Sunile Manjee Can you please clarify what you mean by "full restore" in this case? A truly full restore of HDFS would restore to a cross-cluster consistent Point-in-Time (PiT) snapshot for all datanodes AND namenode. Therefore the namenode's idea of existing blocks would match the blocks existing on the datanodes after the restore. If the PiT snapshot was not perfect (for instance, if some HDFS files were in the midst of being written during the process of taking the snapshot), then you might need to do some cleanup of the blocks for those affected files. But not via a reformat.

In general, a reformat of HDFS erases everything in it, just as with reformat of any file system. So it seems unlikely this is what you want. If you really do want a reformat, then there is no need to do a restore first; it is all going away anyway.

How was your snapshot taken, and how are you doing the full restore?

Re: Name Node Reformat after hdfs restore

pminovic — Fri, 08 Apr 2016 13:59:25 GMT

Hi @Sunile Manjee, have you seen the answers? Please consider to accept/upvote the helpful ones.