Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Contributor

Cluster upgrade is a well-planned maintenance activity. Before starting the upgrade, it is important to ensure that your cluster is ready for upgrade. Ambari has built-in pre-checks that are executed before the actual upgrade to check the overall health of your cluster and validate its readiness for upgrade. Let us look at the pre-checks, their purpose and how to remediate the issues, so that your cluster upgrade process is smooth. A bit of background first:

Background

Ambari supports two kinds of stack (cluster) upgrades: Rolling Upgrade (RU) and Express Upgrade (EU). Pre-checks apply to both upgrade methods, although RU has a larger set of pre-checks since the upgrade does not involve any downtime

First up, the types of pre-checks:

  • Cluster level checks: Example – newer version packages should be installed on all hosts in the cluster
  • Service level checks: Example – all service components should be in installed state (no failures)
  • Host level checks: All hosts must be communicating with Ambari (no network issues, no heartbeat failures from Ambari agent to server)

The output of a pre-check would result in three possible values of pre-check status:

  • ERROR
  • WARNING
  • PASS

ERRORS that have to be fixed before the upgrade can be started. WARNINGS are recommended to be fixed, but may be ignored, if you know what you are doing

When you hit ‘Upgrade’ button in Ambari User Interface (UI) – the pre-checks are kicked off in the background and within a matter of seconds you would be notified if there are some issues that need to be fixed before upgrade (RU or EU) can be started. Here is a snippet from Pre-Checks Window in Ambari UI:

7362-pc-image.png

Behind the scenes, below API call is triggered:

curl -u $user:$password -i -H 'X-Requested-By: ambari' -X GET http://$server:8080/api/v1/clusters/$name/rolling_upgrades_check?fields=*&UpgradeChecks/repository_v...

- $user: Ambari admin user

- $password: Password of the Ambari admin user account

- $server: Ambari server hostname or IP address

- $name: Cluster name

- $version: The actual stack version you wish to upgrade to; example: 2.5.0.0-1245 (this must be registered beforehand and package installation should be completed)

- upgrade_type: ROLLING for Rolling Upgrade, NON_ROLLING for Express Upgrade

============================================================================

Details about the pre-checks and remediation

Refer ru-eu-prechecks-details.pdf

Table above is updated as of Ambari 2.4 release

566 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 10:23 AM
Updated by:
 
Contributors
Top Kudoed Authors