Background Starting with Ambari-2.6.1 release, a new repository
has been introduced called HDP-GPL. This article covers FAQs related to the purpose of this repository,
when it takes effect and the impact it can have on your cluster esp. when
dealing with LZO compression libraries. FAQs What is the need for a new GPL repo? LZO compression libraries use a GPL license, which differs from rest of
the packages in Ambari/HDP which are Apache licensed. Hence, the need for a
separate repo which is named ‘HDP-<version_number>-GPL’. The Version
Definition File (VDF) starting HDP-2.6.4.0 release has been updated to include
the new ‘HDP-2.6-GPL’ repo. Refer the sample VDF file for CentOS7 here. Here is a snippet from Ambari web UI for the new 'HDP-2.6-GPL' field: What are the exact contents of HDP-GPL repo? As of HDP-2.6.4.0 release, the repo contains hadooplzo package files as follows: hadooplzo_2_6_4_0_91-0.6.0.2.6.4.0-91.x86_64.rpm hadooplzo_2_6_4_0_91-native-0.6.0.2.6.4.0-91.x86_64.rpm How is Ambari configured to download and install LZO
libraries? For Fresh installs: While running ‘ambari-server setup’, you would be prompted
as follows: Enable Ambari Server to download and install GPL Licensed
LZO packages [y/n] (n)? Depending on your response, a new entry would be created in
ambari.properties file on Ambari server host: ‘gpl.license.accepted=true|false’ For Ambari Upgrades: If you are upgrading to Ambari-2.6.1 (or higher) and have
LZO enabled, you would be prompted to accept GPL license during the ‘ambari
server upgrade’ step [root@myhost yum.repos.d]# ambari-server upgrade
Using python /usr/bin/python
Upgrading ambari-server
INFO: Upgrade Ambari Server
INFO: Updating Ambari Server properties in ambari.properties ...
INFO: Updating Ambari Server properties in ambari-env.sh ...
WARNING: Original file ambari-env.sh kept
INFO: Fixing database objects owner
Ambari Server configured for Embedded Postgres. Confirm you have made a backup of the Ambari Server database [y/n] (y)? y
INFO: Upgrading database schema
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Your cluster is configured to use LZO which is GPL software. You must agree to enable Ambari to continue downloading and installing LZO [y/n] (n)? y
INFO: Return code from schema upgrade command, retcode = 0
INFO: Console output from schema upgrade command:
INFO: {"lzo_enabled":"true"}
INFO: Schema upgrade completed
Adjusting ambari-server permissions and ownership...
Ambari Server 'upgrade' completed successfully.
For both fresh install and upgrades, enabling LZO GPL license is
optional For Express/Rolling Upgrades: Before starting Express/Rolling Upgrade, a pre-check would be performed by Ambari to detect the existence of LZO configs in core-site. In case the GPL license is not accepted and LZO is enabled, you would see the following warning prior to start of Upgrade: You have LZO codec enabled in the core-site config of your cluster. You have to accept GPL license during ambari-server setup to have LZO installed automatically. If any hosts require LZO, it should be installed before starting the upgrade. Consult Ambari documentation for instructions on how to do this. How can I enable LZO GPL license afterwards? ambari-server setup --enable-lzo-under-gpl-license and then restart ambari server service For details on LZO, refer official Ambari documentation here How can I know if GPL license is accepted by Ambari? On Ambari server host, execute below command grep "gpl.license.accepted" /etc/ambari-server/conf/ambari.properties An output of 'gpl.license.accepted=true' implies the license was accepted I do not see the HDP-GPL repo in Manage versions page on
Ambari web UI. Is that expected? Yes, this would be due to non-usage of LZO compression libraries on your
cluster. To confirm, verify if GPL license was denied by Ambari (gpl.license.accepted=false in ambari.properties file) I use a local repository. Does GPL need additional configuration
in such cases? The configuration steps for HDP-GPL local repo are identical
to that of HDP and HDP-UTILS repo. For instance, you would need to first
download the hdp.gpl.repo or the gpl repository
tarball to the selected mirror server. Thereafter, synchronize the repository contents to your mirror server followed by generating the repository metadata: reposync -r HDP-GPL-2.6.4.0
createrepo <web.server.directory>/hdp/<OS>/HDP-GPL-2.6.4.0
For further details on using local repository, refer official
Ambari documentation here What happens when you try to configure core-site with LZO
configs and GPL license is not accepted? Ambari’s stackadvisor has a check to validate such configs. In the current scenario, you would get the following error message: How about some Troubleshooting? ==== Troubleshooting ==== Scenario: Post HDP upgrade, you observe the following error during Pig/MapReduce service check: ERROR 2118: Compression codec com.hadoop.compression.lzo.LzoCodec not found. Cause: Your have LZO configured, but not accepted the GPL
license during Ambari upgrade Resolution: Enable GPL license using "ambari-server setup --enable-lzo-under-gpl-license" command ==== Bug Fixes ==== AMBARI-22906 - Fixed in Ambari-2.6.1.5 release AMBARI-23055 - Fixed in the upcoming Ambari-2.6.2.0 release
... View more
