Member since
11-30-2021
2
Posts
0
Kudos Received
0
Solutions
12-01-2021
08:22 AM
Any suggestions please
... View more
11-30-2021
02:12 AM
Hi All,
I have some huge text and parquet tables. As a part of retention, we are dropping old partitions as per retention policy which are not required anymore. After dropping partitions, executing a compute incremental stats on whole table is executed. As it is on whole table, it is getting failed in impala as it is consuming high HDFS_IO_READ/
My doubts are
1. Do we need to run the compute incremental stats on table to calculate the latest stats or as we are dropping partitions, will those stats (for removed partitions) will be automatically cleared ? 2. Will there be any impact on the performance if we don't execute compute incremental stats?
Impala Version : 3.4.0
Thanks
... View more
Labels:
- Labels:
-
Apache Impala
-
HDFS