- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
query on partitioning
- Labels:
-
Apache Hive
Created 03-23-2023 06:54 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
when we rename partition of a column, would underlying data of the column would also change as per our rename. this is on ORC file format.
for ex. I've partition named activeFlag='N', when I rename this partition to activeFlag='Y', would those activeFlag column value change to 'Y' (OR) data would be intact as 'N' only?
Created on 03-23-2023 03:30 PM - edited 03-23-2023 03:31 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @dearvenkat ,
Sorry I was thinking that you just want to change the value, but if you are using RENAME PARTITION sql command it will depend on which type of table you have
If it's EXTERNAL the data in HDFS will remain in the same old directory.
If your table is MANAGED it should change the directory name to the new one.
Please refer to this article:
https://community.cloudera.com/t5/Support-Questions/Partition-rename-in-Hive-HDFS-path/td-p/193283
Apache Hive SQL Reference:
Created 03-23-2023 08:59 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@dearvenkat Welcome to the Cloudera Community!
To help you get the best possible solution, I have tagged our Hive experts @JoseManuel and @Shmoo who may be able to assist you further.
Please keep us updated on your post, and we hope you find a satisfactory solution to your query.
Regards,
Diana Torres,Community Moderator
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Created 03-23-2023 09:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @dearvenkat , The old data will not be affected by this change.
The column you have chosen to be partition of the table is not changing, what is changing is the internal value from N to Y,
So it will create a new partition and the old data will remain on the partition activeFlag=N.
It is the same if you have a partition by date, every day you will have a new partition and the other days remains on the past partitions.
Hope that helps.
Created 03-23-2023 10:13 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks @Shmoo
Follow-up questions:
1. So, this PARTITION RENAME will only update reference folder paths in Hive metastore?
2. PARTITION RENAME work, if already a partition with activeFlag='Y' exists? If yes, what will happen to existing partition?
Created on 03-23-2023 03:30 PM - edited 03-23-2023 03:31 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @dearvenkat ,
Sorry I was thinking that you just want to change the value, but if you are using RENAME PARTITION sql command it will depend on which type of table you have
If it's EXTERNAL the data in HDFS will remain in the same old directory.
If your table is MANAGED it should change the directory name to the new one.
Please refer to this article:
https://community.cloudera.com/t5/Support-Questions/Partition-rename-in-Hive-HDFS-path/td-p/193283
Apache Hive SQL Reference:
