Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Validate compression type in HDFS

avatar
New Contributor

Hi - i would like to see the compression used in apache parquet and avro file using simple commands in hadoop. can you please guide me...Say for example there is a file in hadoop which is a parquet fie and LZO compression was chosen while creating it by someone else. Now i would like to verify the same in HDFS. How will i do that?

1 REPLY 1

avatar
Master Collaborator

@Sourav1982 

 

You can use parquet-tools and avro-tools command to check the file metadata.

Use the meta option to extract the compression details. 

 

Refer https://github.com/apache/parquet-mr/tree/master/parquet-tools

 

Hope this helps,

Paras

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.