Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to confirm my files are snappy compresses in hive?

avatar
Contributor

So I compressed my table in hive using snappy compression and it did get compress. The size was reduced.

But when i run hadoop fs -lsr /hive/user.db/table_name, I see no file extensions with .snappy.

I want to know if they really were snappy compressed or not?

3 REPLIES 3

avatar
Super Guru

@Akshat Mathur

If you describe the table in hive, you should be able to see the compression algorithm applied to the table:

hive> describe formatted <hive_table>;

I don't believe there is a quick way to see if it is compressed via HDFS.

avatar
Contributor

Yup @Michael Young.

Another way I found was through

hadoop fs -text <file-location>

On the top of results, INFO compress.CodecPool: Got brand-new decompressor [.snappy] is written which I think is a confirmation that snappy compression is applied.

avatar
Super Guru
@Akshat Mathur

That is handy, thank you for sharing! If you think my response was helpful, please accept the answer to make it easier for others to find answers.