Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to Compress Avro/Parquet File in Hive

Highlighted

Unable to Compress Avro/Parquet File in Hive

New Contributor

Hi, I am planning to give exam on Tuesday. 

 

I am currently having problem in compressing Avro/Parquet tables in Hive. I read on documentation that i have to set the following properties i.e.

 

SET hive.exec.compress.output=true

SET parquet.compression = Snappy //for parquet

SET avro.output.codec = Snappy //for avro

 

Here is DDL: 

CREATE TABLE avro_orders(order_id INT, order_Date bigint, order_customer_id INT, order_Status String) STORED AS AVRO;

 

I have also added to --auxpath

/usr/lib/hive/lib/snappy-java-1.0.4.1.jar

 

However i am still not able to achieve compression with the use of these properties. Can anyone give me a clue abut what i am doing wrong ?

 

I am facing similar problem with sqoop. I can't compress Avro/Parquet files however i can compress text files.

 

Thanks in advance