Rolling upgrade from HDF-3.0.2.0 to 3.1 In HDF 3.1 the concept of Rolling Upgrade will be introduced for the first time. Prior to this release, HDF only supported Express Upgrade which orchestrated a stop all, upgrade and start all approach to upgrades.Rolling Upgrade orchestrates the upgrade in an order that is meant to preserve cluster operation and minimize service impact during upgrade. This process has more stringent prerequisites (particularly regarding cluster high availability configuration) and can take longer to complete than an Express Upgrade. In the case of HDF, we have components that do not need to be brought down while upgrading HDF from older to newer version. These include Storm (long running topologies), Kafka (data being ingested into topics or topics being accessed by other services), Streaming Analytics Manager (Live streaming analytics applications), Schema Registry (shared repository of schemas to allow application to interact with each other) t Therefore from HDF 3.0.2.* onwards, rolling upgrade support is provided from HDF 3.0.2.* version to 3.1.*. This article will walk readers through an example of executing rolling upgrade as well as downgrade between HDF 3.0.2 and HDF 3.1. The following is a list of all components available in HDF 3.0.2.0.*. For purpose of this example , the cluster will have installed all of them and kerberos will be enabled Zookeeper Storm Ambari Infra Ambari Metrics Kafka Log Search Ranger Nifi Registry Streaming Analytics Manager HDF 3.1 supports Rolling upgrade for all of the above components except NiFi. For NiFi, all flows must be stopped before upgrading. During upgrade for each component, Ambari will manage any configuration changes that are required, similar to Express Upgrade, in order to move from HDF 3.0.2.0 to 3.1.0.0. Ambari Infra, Ambari Metrics and Log Search are upgraded manually in separate steps (see Ambari documentation for details) . Prerequisites to Rolling upgrade: Before beginning the upgrade process, please perform the following actions: P1. Download and upgrade your current mpack to the latest version (3.1.0). This can be done via the command line as shown below: ambari-server stop ambari-server upgrade-mpack --mpack=<hdf-ambari-mpack-file> --verbose ambari-server start P2. Using the Ambari UI download and install the required packages choosing respective OS. From the upper right menu choose → Admin → Stack and Versions → Manage Versions (on left side) → Register Version → Accept Current version or add new version → Save. Later click "install" to install all required rpm packages on all nodes. P3. Once you click “Install”, it will start installing all packages and show a popup as below. You can click on “Installing” link to see progress. Lets begin Rolling Upgrade as demonstrated below. Select the → Admin → Stack and Versions option to see the “Upgrade” button as show below. Select "Show Details" link to verify that respective packages are installed on nodes before beginning Rolling upgrade. 2. Click Upgrade. You will be presented with two options: Rolling Upgrade and Express Upgrade as shown. 3. If any checks notification appears, they can be reviewed using the adjacent link. In the above case for Rolling Upgrade “1 Required 1 Warning” items are alerted. The “Required” query has to be resolved before proceeding as shown below. In this example Ambari is asking to disable Auto Start. To resolve thi, click “Cancel” on your RU pop up. Go to the upper right corner → Admin → Service Auto Start → Click on “Enabled” button. It will change color to disabled. Save it and repeat Step 1 and 2. 4. Now click on “Rolling” Section to begin Rolling upgrade and Click Proceed. 5. Click Yes. If you see any warnings, Please check it and see if you would like to proceed further. 6. Ambari will begin Rolling Upgrade process from 3.0.2.0 to 3.1.* a). Rolling Upgrade begins with the “Prepare Backups” stage where it will prompt user to take backups of databases for various installed components . Backup strategy is different for different databases so please talk to your DB admin before proceeding. b). If Ranger is installed, Ambari will recommend to take “Ranger Admin database” backup. Ranger Backup Prompt (if installed) At any manual step (as shown above) users can choose to downgrade back to older version after clicking “Downgrade” button. This will restore cluster back to HDF 3.0.* version. Note: Please do not edit any SAM’s application or Schema Registry’s schema while Rolling upgrade is in progress. Streaming Analytics Manager (SAM) Database Prompt (if installed) Schema Registry (Registry) Database Prompt (if installed) In the case of SAM and Schema Registry if Downgrade is selected past the database backup prompt then users will have to restore the backup of Streaming Analytics Manager’s and Schema Registry's database respectively mentioned in above step. c). Rolling upgrade of Kafka. (No manual intervention is required) d). Rolling upgrade of Storm. (No manual intervention is required) e). Rolling upgrade of Schema Registry. (No manual intervention is required) f). Rolling upgrade of Streaming Analytics Manager (No manual intervention is required) g). Stop, upgrade config, start for Nifi. (No manual intervention is required) h). If Ranger is installed, Rolling upgrade of Ranger (No Manual intervention is required). 6.8. RU finished. Choose either of them. Downgrade, Finalize Later, Finalize. 6.8.1 Choosing “Downgrade” will downgrade your cluster back to 3.0.2.0 version. Please scroll down to check details. 6.8.2 Choosing “Finalize Later” gives you an ability to use 3.1.* version while still choosing to downgrade later on. 6.8.3 Choosing “Finalize” will finalize the current version to 3.1.*. User cannot go back to any previous version. If a user decides to downgrade the cluster before finalizing the following steps would take place in Ambari: 7.0 Downgrade of HDF cluster from 3.1.* to 3.0.2.0. Click Downgrade. a). Downgrade begins with Ambari stopping Nifi Services and downgrading them . Later Ambari performs a rolling downgrade of Ranger if installed. b). Restore database of Streamline Analytics Manager based on copy we made earlier. Later step will do downgrade of Streamline Analytics server (No manual intervention required). c). We restore database of Schema Registry based on copy we made earlier. Please note, in case of Schema Registry, you will still have a downtime of few seconds until and unless both instances of registry (in case of HA) were managed by load balancer. d). Rolling downgrade of Storm and Kafka e). Downgrade of Zookeeper and Kerberos Descriptors. Downgrade is finished now. ... View more