Created 07-28-2017 06:28 AM
I'm using HDP 2.5.3.
Hive Setting
ACID Transacctions ON
Execution Engine TEZ
CBO ON
Fetch column stats at compiler ON
Default ORC Stripe Size
64MB
ORC Compression Algorithm ZLIB
ORC Storage Strategy SPEED
Here's my question.
If I created hive table like this,
CREATE TABLE test01
no int,
id string,
code, string
ROW FORMAT DELIMITED
FIELDS TERMININATED BY '|'
STORED AS ORC
then, what is default tableproperties of test01 table's ORC options?
TBLPROPERTIES (
'orc.compress' = '?',
'orc.create.index'='?',
'orc.stripe.size'='?',
'orc.row.index.stride'='?'
)
for example.
TBLPROPERTIES (
'orc.compress' = 'ZLIB',
'orc.create.index'='true',
'orc.stripe.size'='67108864',
'orc.row.index.stride'='50000'
)
Created 07-28-2017 07:39 AM
I verified in-house, it does not pick up any value at the table level:
hive> desc formatted test01;
OK
# col_name data_type comment
no int
id string
code string
# Detailed Table Information
Database: default
Owner: hive
CreateTime: Fri Jul 28 07:37:06 UTC 2017
LastAccessTime: UNKNOWN
Protect Mode: None
Retention: 0
Location: hdfs://s:8020/apps/hive/warehouse/test01
Table Type: MANAGED_TABLE
Table Parameters:
COLUMN_STATS_ACCURATE{\"BASIC_STATS\":\"true\"}
numFiles 0
numRows 0
orc.compress ?
orc.create.index ?
orc.row.index.stride?
orc.stripe.size ?
rawDataSize 0
totalSize 0
transient_lastDdlTime1501227426
Created on 07-28-2017 02:23 PM - edited 08-17-2019 10:36 PM
I'm just interesting.
Why does it setup these options in hive config from ambari web?
Well, actually, that means is if I want to use hive ORC file format with advanced TBLPROPERTIES such as "orc.compress, orc.compress.size, orc.stripe.size, orc,create.index....etc", I have to specify these tblproperties options every times when I'm trying to create hive table ORC file format.