Reply
Highlighted
New Contributor
Posts: 4
Registered: ‎05-08-2017

Unable to Compress Avro/Parquet File in Hive

Hi, I am planning to give exam on Tuesday. 

 

I am currently having problem in compressing Avro/Parquet tables in Hive. I read on documentation that i have to set the following properties i.e.

 

SET hive.exec.compress.output=true

SET parquet.compression = Snappy //for parquet

SET avro.output.codec = Snappy //for avro

 

Here is DDL: 

CREATE TABLE avro_orders(order_id INT, order_Date bigint, order_customer_id INT, order_Status String) STORED AS AVRO;

 

I have also added to --auxpath

/usr/lib/hive/lib/snappy-java-1.0.4.1.jar

 

However i am still not able to achieve compression with the use of these properties. Can anyone give me a clue abut what i am doing wrong ?

 

I am facing similar problem with sqoop. I can't compress Avro/Parquet files however i can compress text files.

 

Thanks in advance