- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Validate compression type in HDFS
- Labels:
-
HDFS
Created ‎05-26-2020 07:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi - i would like to see the compression used in apache parquet and avro file using simple commands in hadoop. can you please guide me...Say for example there is a file in hadoop which is a parquet fie and LZO compression was chosen while creating it by someone else. Now i would like to verify the same in HDFS. How will i do that?
Created ‎05-27-2020 04:06 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can use parquet-tools and avro-tools command to check the file metadata.
Use the meta option to extract the compression details.
Refer https://github.com/apache/parquet-mr/tree/master/parquet-tools
Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
