Member since
10-30-2018
3
Posts
1
Kudos Received
0
Solutions
05-08-2019
11:52 AM
Hi Cloudera Community , How i can change the compression codec of hive at runtime. I'm reading some table on avro format compressed with snappy and i'm triying to write a similiar table compressed on snappy but the result is compressed on "deflate", after try with multiple options the resulting files were compressed with the same codec. Can you help me to identify my issue on the following sentences, or what can i do to define the compression codec of hive at runtime. "set hive.exec.compress.output=true; SET mapred.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec; SET mapred.output.compression.type=BLOCK; SET hive.exec.dynamic.partition.mode=nonstrict; CREATE external table IF NOT EXISTS tableX partitioned by (year Int) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' TBLPROPERTIES ('avro.schema.url'='hdfs:///AAA/BBB/CCC/tableX.avsc'); alter table tableX add if not exists partition (year = 2016) location 'hdfs://nameservice/AAA/BBB/CCC/2016'; insert overwrite table tableX partition (year = 2016) SELECT id, name, email FROM tablaY WHERE year = 2016;" Best Regards, Esteban
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache YARN
-
HDFS
04-01-2019
07:16 PM
Hi Cloudera Community, Why Cloudera don't recommended the creation of root.[username] pool on the Resource Pools configuration ? I can't find docummentation realted with this. Best Regards,
... View more
Labels:
- Labels:
-
Apache YARN
-
Cloudera Manager