Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to validate which compression technique has been applied to hive?

avatar
Contributor

I have create table in hive and specified compression as gzip during table creation

I am not able to find way that gzip has been applied on newly created table

is there any command which specify compression technique applied on hive.

1 ACCEPTED SOLUTION

avatar
Contributor

I am able to resolve it. issue with create and select together.

I've created table first then inserted data, now able to see gzip as tableproperties in desc command

View solution in original post

10 REPLIES 10

avatar
@payal patel

The compression of the table can be validated by checking the definition of the hive table using the command:

hive> describe formatted <hive_table>;

avatar
Contributor

I am able to see that compressed as No. I think I need to set configuration for compression separately in hive

avatar
Contributor

able to run ..Thanks

avatar

@payal patel

What you see when you run show create table <tablename>; ?

avatar
Contributor

there is message only that query runs "succeed"

avatar
Contributor

I am running hive query using HIve view of Ambari and executing below query

CREATE TABLE t11 STORED AS parquet TBLPROPERTIES("parquet.COMPRESS"="gzip") AS SELECT * FROM abc;

Here abc table already contain some data

Post execution, it shows that query status as "Succeed"

then I verified data by executing "select * from t11".I am able to see data.

I want to verify that gzip has been applied or not.so I did not find way yet

avatar

show create table t11;

What you get in output?

avatar
Contributor

getting below output

CREATE TABLE `t11`( `organisation` string, `org_skill` string, `emp_name` string, `pageurl` string, `cookie` string, `referurl` string, `time` string, `employeeid` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/t11' TBLPROPERTIES ( 'COLUMN_STATS_ACCURATE'='true', 'numFiles'='1', 'numRows'='55', 'parquet.COMPRESS'='gzip', 'rawDataSize'='440', 'totalSize'='2046', 'transient_lastDdlTime'='1469095673')

avatar
Contributor

I am able to resolve it. issue with create and select together.

I've created table first then inserted data, now able to see gzip as tableproperties in desc command