Support Questions

Find answers, ask questions, and share your expertise

How to confirm my files are snappy compresses in hive?

avatar
Contributor

So I compressed my table in hive using snappy compression and it did get compress. The size was reduced.

But when i run hadoop fs -lsr /hive/user.db/table_name, I see no file extensions with .snappy.

I want to know if they really were snappy compressed or not?

3 REPLIES 3

avatar
Super Guru

@Akshat Mathur

If you describe the table in hive, you should be able to see the compression algorithm applied to the table:

hive> describe formatted <hive_table>;

I don't believe there is a quick way to see if it is compressed via HDFS.

avatar
Contributor

Yup @Michael Young.

Another way I found was through

hadoop fs -text <file-location>

On the top of results, INFO compress.CodecPool: Got brand-new decompressor [.snappy] is written which I think is a confirmation that snappy compression is applied.

avatar
Super Guru
@Akshat Mathur

That is handy, thank you for sharing! If you think my response was helpful, please accept the answer to make it easier for others to find answers.