Reply
Explorer
Posts: 6
Registered: ‎05-29-2016

Is it possible to backup the HDFS metadata without stopping the cluster (in a ha cluster)?

Hello,

 

I need to backup the HDFS metadata but I must not stop the cluster due to long running jobs. Is it possible to make this backup just by stopping the roles on the inactive namenode and make the backup? Will this backup be consistent? It's about a high availability cluster. 

 

In the documentation it says "This backup method requires you to shut down the cluster." - so I guess there are some other ways too. The link: 

http://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_mc_hdfs_metadata_backup.html

 

Thanks,

D. 

 

 

Contributor
Posts: 52
Registered: ‎10-19-2015

Re: Is it possible to backup the HDFS metadata without stopping the cluster (in a ha cluster)?

If you stop namenode(HDFS), your backup should be legimate. But if you do shutdown namenode, your running jobs should fail as a result. So it is still better to shtudown the whole cluster.

 

I agree that if you shutdown the mapredue or YARN cluster, you may still have orphan tasks hanging on slave nodes for  a long time. In my expereice, you can use massh or what evertool to  find and to kill those orphan jobs manually

Announcements