Member since
07-15-2022
11
Posts
0
Kudos Received
0
Solutions
08-18-2022
05:18 PM
Hi all, I have generated a query and the summary of Memory as below. Need your help to elaborate and how to read the summary. Thank you. Query(9c4786bd80e8ed20:1caa788500000000): Limit=3.00 GB Reservation=0 ReservationLimit=2.40 GB OtherMemory=0 Total=0 Peak=520.47 MB
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
08-17-2022
07:42 PM
Hi all, I am new to Hive, was told below parameter used to improve hive performance, if i were to set and run those code sequence as below, does the sequence matters and correlated to each other? Do we need to put up these code whenever run the query? Or execute once will be sufficient? Thanks. Set hive.exec.parallel = true;
set hive.vectorized.execution.enabled = true;
set hive.vectorized.execution.reduce.enabled = true;
set hive.cbo.enable=true;
set hive.compute.query.using.stats=true;
set hive.stats.fetch.column.stats=true;
set hive.stats.fetch.partition.stats=true;
set mapred.compress.map.output = true;
set mapred.output.compress= true;
SET hive.exec.dynamic.partition = true;
SET hive.exec.dynamic.partition.mode = nonstrict;
set hive.auto.convert.join = false;
set hive.exec.max.dynamic.partitions=100000;
set hive.exec.max.dynamic.partitions.pernode=10000;
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
07-21-2022
02:39 AM
Hi @EricL, If we are using company laptop and not allowed to place these files, is there any other solutions? Thanks.
... View more
07-17-2022
03:58 AM
Hi @colorsoflife Consider using CTE is possible, with the sequence as below for your references. This is Hive script incorporate into Ozzie workflow. set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
with test_CTE as
(select * from table1),
testone_CTE as
(select col1, col2, col3 from test_CTE)
insert into table mytablename partition(biz_dt)
select col1 as name1, col2 as name2, col3 as name3 from testOne_CTE
... View more
07-17-2022
01:30 AM
Hi @csguna , set hive.variable.substitute=true; Could you share what is the purpose inserting this line to the code? Thanks.
... View more
07-15-2022
07:35 PM
Hi folks, Can I use the below script in Ozzie workflow? WITH order_data (order_date, product_name, amount) AS ( SELECT order_date, product_name, SUM(i.quantity * i.list_price * (1 - discount)) FROM sales.orders o INNER JOIN sales.order_items i ON i.order_id = o.order_id INNER JOIN production.products p ON p.product_id = i.product_id GROUP BY order_date, product_name ) SELECT * FROM order_data;
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
-
Apache Oozie
07-15-2022
07:25 PM
Could help to explain what's the meaning of first 2 sentence? Set Hive..... What would happen if we don't have 2 sentences? How would that impact our query? Thanks.
... View more