Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to confirm my files are snappy compresses in hive?

How to confirm my files are snappy compresses in hive?

New Contributor

So I compressed my table in hive using snappy compression and it did get compress. The size was reduced.

But when i run hadoop fs -lsr /hive/user.db/table_name, I see no file extensions with .snappy.

I want to know if they really were snappy compressed or not?

3 REPLIES 3
Highlighted

Re: How to confirm my files are snappy compresses in hive?

@Akshat Mathur

If you describe the table in hive, you should be able to see the compression algorithm applied to the table:

hive> describe formatted <hive_table>;

I don't believe there is a quick way to see if it is compressed via HDFS.

Re: How to confirm my files are snappy compresses in hive?

New Contributor

Yup @Michael Young.

Another way I found was through

hadoop fs -text <file-location>

On the top of results, INFO compress.CodecPool: Got brand-new decompressor [.snappy] is written which I think is a confirmation that snappy compression is applied.

Re: How to confirm my files are snappy compresses in hive?

@Akshat Mathur

That is handy, thank you for sharing! If you think my response was helpful, please accept the answer to make it easier for others to find answers.

Don't have an account?
Coming from Hortonworks? Activate your account here