Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Check compressed Hive tables on cluster

New Contributor

Hello guys,

 

I am try to get a information about the compressed tables present on my CDH 5.8.3. To do that, I use "describe extended formatted" tool with Hue, Beeline and Hive cli but always returns the same output:

 

Compressed:         No     

 

But this is fake because the table is compressed. I found one Jira about this problem https://issues.apache.org/jira/browse/HIVE-2250 , but the problem is unresolved yet. Next, I checked the Hive metastore tables and the compressed value is the same fake.

 

Do you know any way to get the tables compressed?

 

Thanks in advance.

 

 

2 REPLIES 2

Champion

Did you had a chance to see the below values in Hive / mapreduce properties 

 

 hive.exec.compress.output=
 mapreduce.output.fileoutputformat.compress=
 mapreduce.output.fileoutputformat.compress.codec=   
 mapreduce.output.fileoutputformat.compress.type= 

New Contributor

Thank you for the response csguna.

The value of this properties now is:

hive.exec.compress.output=true
mapreduce.output.fileoutputformat.compress=false
mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec
mapreduce.output.fileoutputformat.compress.type=BLOCK

 

I am trying to obtain the information with "show create table" statement but this method return the compress format if you specify a value in tblproperties param:

 

CREATE TABLE testsnappy_orc STORED AS ORC TBLPROPERTIES("orc.compress"="snappy") AS SELECT * FROM sourcetable;

 

However if you set the value of compression previously and not add the tblproperties param:

 

 

set parquet.compression=SNAPPY;
CREATE TABLE testsnappy_pq STORED AS PARQUET AS SELECT * FROM sourcetable;

 

 

When you execute "show create table testsnappy_pq" you can't view anything about the compress format.

 

Now, I use the first method to view Hive compressed tables but in my opinion this is not a good way to obtain the information, because you not cover all cases no?.

 

Thank you.

 

 

 

 

 

 

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.