Support Questions
Find answers, ask questions, and share your expertise

Apply snappy compression

Apply snappy compression

 
3 REPLIES 3

Re: Apply snappy compression

Expert Contributor

Hi @Gayathri Devi,

The operation of compression/decompression will increase the cpu load of around 5-10 %

For the memory, is will decrease the disk space by around 70%, more over the size on the disk will be smaller then you will need less iops. Because of that you should see a general improvement of your performance.

Michel

Re: Apply snappy compression

@msumbul

For my use case can you suggest any ideas pls? thanks.

Use case is millions of records in Hbase. Stored in Hbase. have a external table in hive pointing to Hbase. Need to write Impala queries for retrival. For processing data in Hbase i am using hive queries.

Re: Apply snappy compression

Expert Contributor

Hi @Gayathri Devi,

I can't give you more idea than in my previous comment. Because It depends on the system specification that you have, the other load on the cluster, size of the data. size of each lines, etc...

The % that I gave you is based on benchmark that I made in previous project and blog/forum that I read in the past.
The best that you can do is a test. I would recommend you to do a test with compression and another without to see the impact that it have on your environnement.

Moreover, be careful with Hive on top of hbase. You might have bad performance because often it start a full scan of the hbase table, which is an expensive operation.

Michel