Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Snappy Installation on HDP


I have a requirement to compress the HBase tabes in a HDP cluster using snappy. However, I have some clarifications over Snappy installation on HDP.

I executed the Compression Test even without installing snappy and I got a success output. I tried both the commands below.

hbase org.apache.hadoop.hbase.util.CompressionTest file:///tmp/test.txt snappy
hbase org.apache.hadoop.hbase.util.CompressionTest hdfs://hbase.primary.namenode:8020/tmp/test1.txt snappy

On both the occasions the output was:

2017-10-30 11:25:18,454 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-10-30 11:25:18,671 INFO  [main] compress.CodecPool: Got brand-new compressor [.snappy]
2017-10-30 11:25:18,679 INFO  [main] compress.CodecPool: Got brand-new compressor [.snappy]
2017-10-30 11:25:21,560 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-10-30 11:25:22,366 INFO  [main] compress.CodecPool: Got brand-new decompressor [.snappy]

I see below libraries in the path /usr/hdp/ as well.


So my questions are:

Does HDP support snappy compression by default? Don't we need to install Snappy?

Please verify this. The HDP version I'm using is 2.6.1.


Super Mentor

@Thanuja Kularathna

As per the doc of HDP you should install snappy-devel packages on all hosts.

----For RHEL/CentOS/Oracle Linux:
yum install snappy snappy-devel

----Execute the following command at all the nodes in your cluster:
 RHEL/CentOS/Oracle Linux:
yum install lzo lzo-devel hadooplzo hadooplzo-native


Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.