- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Drop partition remove data but not HDFS folder
- Labels:
-
Apache Hive
Created on ‎03-25-2019 12:49 AM - edited ‎09-16-2022 07:15 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We have CDH 5.15 and when we drop partition in Hive it removes the partition and its data but the folder of the partition remains empty and is not removed.
How can I change this behavior?
Thanks
Created ‎03-26-2019 07:53 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If it is a External table we need to delete the directory manually.
ALTER TABLE table_name DROP [IF EXISTS] PARTITION <>;
hadoop fs -rm -r <partition file path>
Thanks
Jerry
Created ‎03-27-2019 12:14 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
It's an internal table with 4 levels of partitions,
But still, when I remove the partition the data and metadata is deleted but not the folder.
Thanks
Created ‎03-31-2019 12:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I assume that you work on the managed table instead of external table? This could be because of lack of permissions from the user who tried to run the DROP command to remove the underlining HDFS path. Check HMS server log to see if you can find any error messages.
