Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Compression LZO

Compression LZO

New Contributor

Hi everyone,

Is anyone performed LZO compression on Hortonwork platform? if so kindly share your experience as am not able to perform it. The reason is I couldn't find the libraries to install and even don't know the complete procedure... Any help appreciated in this regard.

Best Regards

Muhammad Touseef

7 REPLIES 7

Re: Compression LZO

Mentor

@Muhammad Touseef

here you go http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_ambari_reference_guide/content/ch_amb_ref... pretty standard approach, please use the documentation I provided.

Re: Compression LZO

New Contributor

Thanks Artem,

I didn't find any libraries for LZO - as you know it doesn't come generally with the Hadoop as gzip and bzip comes therefore if you could give the rights rpms and jars files so i can enable lzo on the linux system to start compression

Re: Compression LZO

Mentor

Re: Compression LZO

New Contributor

yes in this reference only zypper command to install lzo, lzop etc. i need rpm one to install first on linux which is currently am failed to find it.

Re: Compression LZO

Mentor

3.2. Install LZO

Execute the following command at all the nodes in your cluster:

  • RHEL/CentOS/Oracle Linux:

    yum install lzo lzo-devel hadooplzo hadooplzo-native

Highlighted

Re: Compression LZO

Contributor

@Muhammad Touseef - Assuming you are using HDP-2.6.4.0 and Ambari-2.6.1 version, is GPL license enabled in your Ambari server? To verify, execute below command on Ambari server host:

> grep gpl /etc/ambari-server/conf/ambari.properties

If the output says: gpl.license.accepted=true

Then GPL is enabled, else it isn't. If GPL is not enabled, run below command from Ambari server host:

ambari-server setup --enable-lzo-under-gpl-license

Once GPL is enabled, refer Ambari docs related to LZO: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-administration/content/ch_amb_ref...

Hope this helps!

- Vivek

Re: Compression LZO

Contributor

@Muhammad Touseef - Assuming you are using Ambari-2.6.1 and HDP-2.6.4, is GPL license accepted during Ambari server setup. To verify on Ambari server host, execute below command

> grep gpl /etc/ambari-server/conf/ambari.properties

If you see an output as 'gpl.license.accepted=false' ; GPL is not accepted.

To enable GPL, run:

ambari-server setup --enable-lzo-under-gpl-license

Thereafter refer docs to enable LZO: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-administration/content/ch_amb_ref...

Hope this helps!

- Vivek