question Re: query on partitioning in Support Questions

query on partitioning

dearvenkat — Thu, 23 Mar 2023 13:54:15 GMT

when we rename partition of a column, would underlying data of the column would also change as per our rename. this is on ORC file format.

for ex. I've partition named activeFlag='N', when I rename this partition to activeFlag='Y', would those activeFlag column value change to 'Y' (OR) data would be intact as 'N' only?

Re: query on partitioning

DianaTorres — Thu, 23 Mar 2023 15:59:06 GMT

@dearvenkat Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Hive experts @JoseManuel and @Shmoo who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Re: query on partitioning

Shmoo — Thu, 23 Mar 2023 16:22:22 GMT

Hi @dearvenkat , The old data will not be affected by this change.

The column you have chosen to be partition of the table is not changing, what is changing is the internal value from N to Y,

So it will create a new partition and the old data will remain on the partition activeFlag=N.

It is the same if you have a partition by date, every day you will have a new partition and the other days remains on the past partitions.

Hope that helps.

Re: query on partitioning

dearvenkat — Thu, 23 Mar 2023 17:13:57 GMT

Thanks @Shmoo

Follow-up questions:

1. So, this PARTITION RENAME will only update reference folder paths in Hive metastore?

2. PARTITION RENAME work, if already a partition with activeFlag='Y' exists? If yes, what will happen to existing partition?

Re: query on partitioning

Shmoo — Thu, 23 Mar 2023 22:31:46 GMT

Hi @dearvenkat ,

Sorry I was thinking that you just want to change the value, but if you are using RENAME PARTITION sql command it will depend on which type of table you have

If it's EXTERNAL the data in HDFS will remain in the same old directory.

If your table is MANAGED it should change the directory name to the new one.

Please refer to this article:

https://community.cloudera.com/t5/Support-Questions/Partition-rename-in-Hive-HDFS-path/td-p/193283

Apache Hive SQL Reference:

https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl#LanguageManualDDL-RenamePartition