Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Snappy Installation on HDP

Highlighted

Snappy Installation on HDP

New Contributor

I have a requirement to compress the HBase tabes in a HDP cluster using snappy. However, I have some clarifications over Snappy installation on HDP.

I executed the Compression Test even without installing snappy and I got a success output. I tried both the commands below.

hbase org.apache.hadoop.hbase.util.CompressionTest file:///tmp/test.txt snappy
hbase org.apache.hadoop.hbase.util.CompressionTest hdfs://hbase.primary.namenode:8020/tmp/test1.txt snappy

On both the occasions the output was:

2017-10-30 11:25:18,454 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-10-30 11:25:18,671 INFO  [main] compress.CodecPool: Got brand-new compressor [.snappy]
2017-10-30 11:25:18,679 INFO  [main] compress.CodecPool: Got brand-new compressor [.snappy]
2017-10-30 11:25:21,560 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-10-30 11:25:22,366 INFO  [main] compress.CodecPool: Got brand-new decompressor [.snappy]
SUCCESS

I see below libraries in the path /usr/hdp/2.6.1.0-129/hadoop/lib/native/ as well.

libhadoop.a 
libhadooppipes.a 
libhadoop.so 
libhadoop.so.1.0.0 
libhadooputils.a 
libhdfs.a 
libsnappy.so 
libsnappy.so.1 
libsnappy.so.1.1.4

So my questions are:

Does HDP support snappy compression by default? Don't we need to install Snappy?

Please verify this. The HDP version I'm using is 2.6.1.

2 REPLIES 2

Re: Snappy Installation on HDP

Super Mentor

@Thanuja Kularathna

As per the doc of HDP you should install snappy-devel packages on all hosts.

----For RHEL/CentOS/Oracle Linux:
yum install snappy snappy-devel

----Execute the following command at all the nodes in your cluster:
 RHEL/CentOS/Oracle Linux:
yum install lzo lzo-devel hadooplzo hadooplzo-native

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_command-line-installation/content/instal...

.

Re: Snappy Installation on HDP

New Contributor
Don't have an account?
Coming from Hortonworks? Activate your account here