Background Starting with Ambari-2.6.1 release, a new repository has been introduced called HDP-GPL. This article covers FAQs related to the purpose of this repository, when it takes effect and the impact it can have on your cluster esp. when dealing with LZO compression libraries. FAQs What is the need for a new GPL repo? LZO compression libraries use a GPL license, which differs from rest of the packages in Ambari/HDP which are Apache licensed. Hence, the need for a separate repo which is named ‘HDP-<version_number>-GPL’. The Version Definition File (VDF) starting HDP-2.6.4.0 release has been updated to include the new ‘HDP-2.6-GPL’ repo. Refer the sample VDF file for CentOS7 here. Here is a snippet from Ambari web UI for the new 'HDP-2.6-GPL' field: What are the exact contents of HDP-GPL repo? As of HDP-2.6.4.0 release, the repo contains hadooplzo package files as follows: hadooplzo_2_6_4_0_91-0.6.0.2.6.4.0-91.x86_64.rpm hadooplzo_2_6_4_0_91-native-0.6.0.2.6.4.0-91.x86_64.rpm How is Ambari configured to download and install LZO libraries? For Fresh installs: While running ‘ambari-server setup’, you would be prompted as follows: Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? Depending on your response, a new entry would be created in ambari.properties file on Ambari server host: ‘gpl.license.accepted=true|false’ For Ambari Upgrades: If you are upgrading to Ambari-2.6.1 (or higher) and have LZO enabled, you would be prompted to accept GPL license during the ‘ambari server upgrade’ step [root@myhost yum.repos.d]# ambari-server upgrade Using python /usr/bin/python Upgrading ambari-server INFO: Upgrade Ambari Server INFO: Updating Ambari Server properties in ambari.properties ... INFO: Updating Ambari Server properties in ambari-env.sh ... WARNING: Original file ambari-env.sh kept INFO: Fixing database objects owner Ambari Server configured for Embedded Postgres. Confirm you have made a backup of the Ambari Server database [y/n] (y)? y INFO: Upgrading database schema GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Your cluster is configured to use LZO which is GPL software. You must agree to enable Ambari to continue downloading and installing LZO [y/n] (n)? y INFO: Return code from schema upgrade command, retcode = 0 INFO: Console output from schema upgrade command: INFO: {"lzo_enabled":"true"} INFO: Schema upgrade completed Adjusting ambari-server permissions and ownership... Ambari Server 'upgrade' completed successfully. For both fresh install and upgrades, enabling LZO GPL license is optional For Express/Rolling Upgrades: Before starting Express/Rolling Upgrade, a pre-check would be performed by Ambari to detect the existence of LZO configs in core-site. In case the GPL license is not accepted and LZO is enabled, you would see the following warning prior to start of Upgrade: You have LZO codec enabled in the core-site config of your cluster. You have to accept GPL license during ambari-server setup to have LZO installed automatically. If any hosts require LZO, it should be installed before starting the upgrade. Consult Ambari documentation for instructions on how to do this. How can I enable LZO GPL license afterwards? ambari-server setup --enable-lzo-under-gpl-license and then restart ambari server service For details on LZO, refer official Ambari documentation here How can I know if GPL license is accepted by Ambari? On Ambari server host, execute below command grep "gpl.license.accepted" /etc/ambari-server/conf/ambari.properties An output of 'gpl.license.accepted=true' implies the license was accepted I do not see the HDP-GPL repo in Manage versions page on Ambari web UI. Is that expected? Yes, this would be due to non-usage of LZO compression libraries on your cluster. To confirm, verify if GPL license was denied by Ambari (gpl.license.accepted=false in ambari.properties file) I use a local repository. Does GPL need additional configuration in such cases? The configuration steps for HDP-GPL local repo are identical to that of HDP and HDP-UTILS repo. For instance, you would need to first download the hdp.gpl.repo or the gpl repository tarball to the selected mirror server. Thereafter, synchronize the repository contents to your mirror server followed by generating the repository metadata: reposync -r HDP-GPL-2.6.4.0 createrepo <web.server.directory>/hdp/<OS>/HDP-GPL-2.6.4.0 For further details on using local repository, refer official Ambari documentation here What happens when you try to configure core-site with LZO configs and GPL license is not accepted? Ambari’s stackadvisor has a check to validate such configs. In the current scenario, you would get the following error message: How about some Troubleshooting? ==== Troubleshooting ==== Scenario: Post HDP upgrade, you observe the following error during Pig/MapReduce service check: ERROR 2118: Compression codec com.hadoop.compression.lzo.LzoCodec not found. Cause: Your have LZO configured, but not accepted the GPL license during Ambari upgrade Resolution: Enable GPL license using "ambari-server setup --enable-lzo-under-gpl-license" command ==== Bug Fixes ==== AMBARI-22906 - Fixed in Ambari-2.6.1.5 release AMBARI-23055 - Fixed in the upcoming Ambari-2.6.2.0 release ... View more

Ambari version 2.5 provides a new Upgrade History page. This is useful in many ways such as: Determine the history of upgrades that your cluster has undergone (and any issues). Collect any specific logs from within the web UI without the need to login to the respective host. Determine time taken for upgrade, which gives an indication of the time required for the next upgrade. In short, Upgrade planning and troubleshooting are two aspects addressed through this page Here is how the ‘Upgrade History’ page looks: Besides the above UI, you could also retrieve the entire Upgrade output through a couple of Ambari API calls http://<ambari_server>:8080/api/v1/clusters/<cluster_name>/upgrades The above call provides the list of Upgrade ids. The highest id in the list is usually the most recent Upgrade on the cluster. This upgrade_id serves as an input to the second call as follows: http://<ambari_server>:8080/api/v1/clusters/<cluster_name>/upgrades/<upgrade_id>?fields=upgrade_groups/upgrade_items/tasks/Tasks/status,upgrade_groups/upgrade_items/tasks/Tasks/command_detail,upgrade_groups/upgrade_items/tasks/Tasks/stderr,upgrade_groups/upgrade_items/tasks/Tasks/host_name The second call provides the entire Upgrade output grouped by Upgrade Groups, Upgrade Items and finally the Upgrade Tasks. You could extend the call by including additional fields as they appear in an Upgrade task output as below: Additionally you could redirect the output of above call to a file and send it over for troubleshooting any issues. ... View more

Cluster upgrade is a well-planned maintenance activity. Before starting the upgrade, it is important to ensure that your cluster is ready for upgrade. Ambari has built-in pre-checks that are executed before the actual upgrade to check the overall health of your cluster and validate its readiness for upgrade. Let us look at the pre-checks, their purpose and how to remediate the issues, so that your cluster upgrade process is smooth. A bit of background first: Background Ambari supports two kinds of stack (cluster) upgrades: Rolling Upgrade (RU) and Express Upgrade (EU). Pre-checks apply to both upgrade methods, although RU has a larger set of pre-checks since the upgrade does not involve any downtime First up, the types of pre-checks: Cluster level checks: Example – newer version packages should be installed on all hosts in the cluster Service level checks: Example – all service components should be in installed state (no failures) Host level checks: All hosts must be communicating with Ambari (no network issues, no heartbeat failures from Ambari agent to server) The output of a pre-check would result in three possible values of pre-check status: ERROR WARNING PASS ERRORS that have to be fixed before the upgrade can be started. WARNINGS are recommended to be fixed, but may be ignored, if you know what you are doing When you hit ‘Upgrade’ button in Ambari User Interface (UI) – the pre-checks are kicked off in the background and within a matter of seconds you would be notified if there are some issues that need to be fixed before upgrade (RU or EU) can be started. Here is a snippet from Pre-Checks Window in Ambari UI: Behind the scenes, below API call is triggered: curl -u $user:$password -i -H 'X-Requested-By: ambari' -X GET http://$server:8080/api/v1/clusters/$name/rolling_upgrades_check?fields=*&UpgradeChecks/repository_version=$version&UpgradeChecks/upgrade_type=ROLLING - $user: Ambari admin user - $password: Password of the Ambari admin user account - $server: Ambari server hostname or IP address - $name: Cluster name - $version: The actual stack version you wish to upgrade to; example: 2.5.0.0-1245 (this must be registered beforehand and package installation should be completed) - upgrade_type: ROLLING for Rolling Upgrade, NON_ROLLING for Express Upgrade ============================================================================ Details about the pre-checks and remediation Refer ru-eu-prechecks-details.pdf Table above is updated as of Ambari 2.4 release ... View more