11-01-2016 12:14 AM
I need to backup the HDFS metadata but I must not stop the cluster due to long running jobs. Is it possible to make this backup just by stopping the roles on the inactive namenode and make the backup? Will this backup be consistent? It's about a high availability cluster.
In the documentation it says "This backup method requires you to shut down the cluster." - so I guess there are some other ways too. The link:
11-01-2016 04:33 PM
If you stop namenode(HDFS), your backup should be legimate. But if you do shutdown namenode, your running jobs should fail as a result. So it is still better to shtudown the whole cluster.
I agree that if you shutdown the mapredue or YARN cluster, you may still have orphan tasks hanging on slave nodes for a long time. In my expereice, you can use massh or what evertool to find and to kill those orphan jobs manually