CDH Upgrade - 'Finalize Metadata Upgrade' effect, rollback/downgrade possibilities

[#1]

Hi!

I've just finished CDH upgrade with help of Cloudera Manager. Do you know what is the result of 'Finalize Metadata Update' NameNode's option after successful upgrade? I've used this option, but it's still enabled. How can I confirm that upgrade is finalized?

Provided that upgrade is not finalized yet - is there a rollback/downgrade path in case of problems with HDFS? Is it available in any configuration or only if rolling upgrade (HA and Cloudera Enterprise in place) is carried out?

[#2]

Hi,

Most likely you have found it out by now, but an easy way is to check whether the NameNode's data directory contains the 'previous' folder in addition to the 'current' folder which is always there during normal operations.

Also in the newer versions Cloudera Manager gives you a warning when you are running with unfinalized metadata.

[#3]

Hello @mbalassi,

thank you very much for your reply, but I still have some doubts regarding
this feature. I've just upgraded CDH5.9 to 5.11 and after that I don't see
any 'previous' directory in Namenodes' Data Directories, neither in
Datanodes data directories.
Cloudera's documentation suggests this action is required only if you
upgrade CDH version 5.0, 5.1 or 5.2:
https://www.cloudera.com/documentation/enterprise/5-11-x/PDF/cloudera-upgrade.pdf
Similarly -upgradeProgress subcommand is unavailable in hadodp dfsadmin
command in my current CDH installation. My guess is that Cloudera doesn't
support this mechanism and keeps it only for compliance with older
versions. Do you think it's correct assumption?

[#4 · Accepted solution]

Hi @szamasz, in the case of the 5.9 to 5.11 upgrade you do not see it, because there is no NameNode metadata format change, so there is no need to keep the previous format metadata around (both of those are on HDFS 2.6.0). If you want to see the change you can see it between 5.3 and 5.4 or upgrading to 6.x from 5.x.

The NameNode UI will give you a warning anyway if you have unfinalized metadata changes. If you do not see that you are in the clear. I have just upgraded a cluster from 5.3 to 5.13 and have not finalized the upgrade, can take you a screenshot tomorrow. 🙂

[#5]

Wow! That's very informative contribution, thank you for solving one of my
long lasting questions about CDH. The screenshot will be helpfull too, not
only for me, but for all community members. Thanks a million!

[#6]

The pdf that you are using is deprecated. We are working on removing it from the server. You should only use the current upgrade docs:

https://www.cloudera.com/documentation/enterprise/upgrade/topics/ug_overview.html