Created on 08-16-2017 05:27 AM - edited 09-16-2022 05:06 AM
Hello guys,
I am try to get a information about the compressed tables present on my CDH 5.8.3. To do that, I use "describe extended formatted" tool with Hue, Beeline and Hive cli but always returns the same output:
Compressed: | No |
But this is fake because the table is compressed. I found one Jira about this problem https://issues.apache.org/jira/browse/HIVE-2250 , but the problem is unresolved yet. Next, I checked the Hive metastore tables and the compressed value is the same fake.
Do you know any way to get the tables compressed?
Thanks in advance.
Created 08-17-2017 08:42 PM
Did you had a chance to see the below values in Hive / mapreduce properties
hive.exec.compress.output= mapreduce.output.fileoutputformat.compress= mapreduce.output.fileoutputformat.compress.codec= mapreduce.output.fileoutputformat.compress.type=
Created on 08-18-2017 01:30 AM - edited 08-18-2017 01:31 AM
Thank you for the response csguna.
The value of this properties now is:
hive.exec.compress.output=true mapreduce.output.fileoutputformat.compress=false mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec mapreduce.output.fileoutputformat.compress.type=BLOCK
I am trying to obtain the information with "show create table" statement but this method return the compress format if you specify a value in tblproperties param:
CREATE TABLE testsnappy_orc STORED AS ORC TBLPROPERTIES("orc.compress"="snappy") AS SELECT * FROM sourcetable;
However if you set the value of compression previously and not add the tblproperties param:
set parquet.compression=SNAPPY; CREATE TABLE testsnappy_pq STORED AS PARQUET AS SELECT * FROM sourcetable;
When you execute "show create table testsnappy_pq" you can't view anything about the compress format.
Now, I use the first method to view Hive compressed tables but in my opinion this is not a good way to obtain the information, because you not cover all cases no?.
Thank you.