I have Cloudera 5.16.1 with HA enabled for HDFS.
I want to upgrade OS on both NameNode servers (Active & Standby) to CentOS 7 and keep all settings.
What's the correct way to do it ?
I tried the following:
1. Backed up jn & nn folders in both NameNode servers along with other few folders
2. Stopped all services in the cluster
3. Deleted standby NameNode server from cluster and from cloudera manager
4. Reinstalled the server
5. Added server back to cluster
6. Restored files
7. Started services back
But, it failed to re-enable the HA again.
Please assist / advise
Did you follow the official Cloudera OS upgrade best practice your method looks strange, I am a bit surprised and wondering all the trouble you'd go through if you had also RM , Hive Metastore HA?
I think you took the wrong approach, the reason you have HA is to always have the service up and running in the event you shut down the Active namenode it would have automatically failed over to the standby and you would have gone ahead and upgraded the OS independently of the CDH binaries and done with the OS upgrade restarting that node would automatically join the cluster and become the standby . After the journalNodes have applied all the logs editlog and merged the FSImage you could forcefully or gracefully failover and proceed with the upgrade of the other node.
If you have enabled high availability for the NameNode or JobTracker, follow this procedure:
The above are the steps you should have taken it's from Cloudera, can you share the document you followed?
Thanks for your response and sorry for the delayed answer.
Your solution did not solve the issue because of the infrastructure our cloudera is installed on it.
But we managed to solve it by backing up the needed folders, recreating the servers with same hostname & IP address but with newer OS and then we restored the folders and everything worked fine.