Support Questions

Find answers, ask questions, and share your expertise

Apache Impala: Is compute incremental stats command need to be executed after dropping partition in a table

avatar
New Contributor

Hi All,

 

I have some huge text and parquet tables.
As a part of retention, we are dropping old partitions as per retention policy which are not required anymore. After dropping partitions, executing a compute incremental stats on whole table is executed. As it is on whole table, it is getting failed in impala as it is consuming high HDFS_IO_READ/

 

My doubts are

1. Do we need to run the compute incremental stats on table to calculate the latest stats or as we are dropping partitions, will those stats (for removed partitions) will be automatically cleared ?

2. Will there be any impact on the performance if we don't execute compute incremental stats?

 

Impala Version : 3.4.0


Thanks

1 REPLY 1

avatar
New Contributor

Any suggestions please