Created 02-27-2018 07:57 AM
So I compressed my table in hive using snappy compression and it did get compress. The size was reduced.
But when i run hadoop fs -lsr /hive/user.db/table_name, I see no file extensions with .snappy.
I want to know if they really were snappy compressed or not?
Created 03-02-2018 01:26 PM
If you describe the table in hive, you should be able to see the compression algorithm applied to the table:
hive> describe formatted <hive_table>;
I don't believe there is a quick way to see if it is compressed via HDFS.
Created 03-02-2018 05:38 PM
Yup @Michael Young.
Another way I found was through
hadoop fs -text <file-location>
On the top of results, INFO compress.CodecPool: Got brand-new decompressor [.snappy] is written which I think is a confirmation that snappy compression is applied.
Created 03-02-2018 05:42 PM
That is handy, thank you for sharing! If you think my response was helpful, please accept the answer to make it easier for others to find answers.