Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

query on partitioning

avatar
New Contributor

when we rename partition of a column, would underlying data of the column would also change as per our rename. this is on ORC file format.

 

for ex. I've partition named activeFlag='N', when I rename this partition to activeFlag='Y', would those activeFlag column value change to 'Y' (OR) data would be intact as 'N' only?

1 ACCEPTED SOLUTION

avatar
Rising Star

Hi @dearvenkat  , 

Sorry I was thinking that you just want to change the value, but if you are using RENAME PARTITION sql command it will depend on which type of table you have

If it's EXTERNAL the data in HDFS will remain in the same old directory.

If your table is MANAGED it should change the directory name to the new one.

 

Please refer to this article:

https://community.cloudera.com/t5/Support-Questions/Partition-rename-in-Hive-HDFS-path/td-p/193283

 

Apache Hive SQL Reference:

https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl#LanguageManualDDL-RenamePartitio...

 

View solution in original post

4 REPLIES 4

avatar
Community Manager

@dearvenkat Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Hive experts @JoseManuel and @Shmoo  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Rising Star

Hi @dearvenkat , The old data will not be affected by this change.

The column you have chosen to be partition of the table is not changing, what is changing is the internal value from N to Y,

So it will create a new partition and the old data will remain on the partition activeFlag=N.

 

It is the same if you have a partition by date, every day you will have a new partition and the other days remains on the past partitions.

 

Hope that helps.

avatar
New Contributor

Thanks @Shmoo 

Follow-up questions:

1. So, this PARTITION RENAME will only update reference folder paths in Hive metastore?

2. PARTITION RENAME work, if already a partition with activeFlag='Y' exists? If yes, what will happen to existing partition?

avatar
Rising Star

Hi @dearvenkat  , 

Sorry I was thinking that you just want to change the value, but if you are using RENAME PARTITION sql command it will depend on which type of table you have

If it's EXTERNAL the data in HDFS will remain in the same old directory.

If your table is MANAGED it should change the directory name to the new one.

 

Please refer to this article:

https://community.cloudera.com/t5/Support-Questions/Partition-rename-in-Hive-HDFS-path/td-p/193283

 

Apache Hive SQL Reference:

https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl#LanguageManualDDL-RenamePartitio...