Member since
12-11-2018
1
Post
0
Kudos Received
0
Solutions
12-11-2018
08:36 PM
Hello, I want to merge the files in a partition in HDFS,
they are results of the same insertion in Hive, executed in different moments. /apps/hive/warehouse/raw.db/XXX_XXXX/part_fecha_proceso=2018-12-05
Permission Owner Group Size Last Modified Replication BlockSize Name
-rwxrwxrwx hive hadoop 54.25KB 11-12-2018 16:48:43 1 128MB 000000_0
-rwxrwxrwx hive hadoop 54.25KB 11-12-2018 16:51:53 1 128MB 000000_0_copy_1 I put this variables before the insert but not work. set hive.execution.engine=tez;
set hive.merge.tezfiles=true;
set hive.merge.smallfiles.avgsize=128000000;
set hive.merge.size.per.task=128000000;
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
SET hive.exec.max.dynamic.partitions=100000;
SET hive.exec.max.dynamic.partitions.pernode=100000; I try this on sandbox HDP, can you help me please? ------------------- Español.. Hola, quiero unir los archivos de una particion en HDFS
resultantes de los mismos inserts en Hive ejecutados en momentos distintos. /apps/hive/warehouse/raw.db/XXX_XXXX/part_fecha_proceso=2018-12-05
Permission Owner Group Size Last Modified Replication BlockSize Name
-rwxrwxrwx hive hadoop 54.25KB 11-12-2018 16:48:43 1 128MB 000000_0
-rwxrwxrwx hive hadoop 54.25KB 11-12-2018 16:51:53 1 128MB 000000_0_copy_1 Coloqué las siguientes variables antes de realizar el
insert, pero no funcionó. set hive.execution.engine=tez;
set hive.merge.tezfiles=true;
set hive.merge.smallfiles.avgsize=128000000;
set hive.merge.size.per.task=128000000;
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
SET hive.exec.max.dynamic.partitions=100000;
SET hive.exec.max.dynamic.partitions.pernode=100000; Lo trate de realizar en el sandbox de HDP, pueden ayudarme
porfavor?
... View more
Labels: