Created on 08-14-2019 12:00 PM - edited 08-14-2019 12:03 PM
Let's assume that I have a table test_tbl which was created through impala-shell.
I have a few questions :
Thanks in advance.
Created 08-14-2019 03:31 PM
REFRESH the table only when I add new data through HIVE or HDFS commands ? That is when I am doing insert into ...through impala-shell no need for refreshing ?
Correct.
- INVALIDATE METADATA of the table only when I change the structure of the table (add columns, drop partitions) through HIVE?
Correct. Or creating new tables through Hive.
- DROPping partitions of a table through impala-shell (i.e alter table .. drop partition .. purge). Do I have to do REFRESH or INVALIDATE METADATA?
No.
- DROPping partitions of a table through impala-shell . How can I compute the new stats of the partitioned table? Compute incremental stats OR Drop Incremental stats before dropping partition ?
The next time you run an incremental stats for a new partition Impala will update things correctly (e.g. the global row count)
Created 08-14-2019 03:31 PM
REFRESH the table only when I add new data through HIVE or HDFS commands ? That is when I am doing insert into ...through impala-shell no need for refreshing ?
Correct.
- INVALIDATE METADATA of the table only when I change the structure of the table (add columns, drop partitions) through HIVE?
Correct. Or creating new tables through Hive.
- DROPping partitions of a table through impala-shell (i.e alter table .. drop partition .. purge). Do I have to do REFRESH or INVALIDATE METADATA?
No.
- DROPping partitions of a table through impala-shell . How can I compute the new stats of the partitioned table? Compute incremental stats OR Drop Incremental stats before dropping partition ?
The next time you run an incremental stats for a new partition Impala will update things correctly (e.g. the global row count)
Created 08-14-2019 05:27 PM