- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to delete/drop a partition of an external table along with data
- Labels:
-
Apache Hive
Created ‎01-25-2017 05:21 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have an external table which is created with partitions and i would like to delete/drop few partition along with data as i no longer require it.
Appreciate any suggestions.
Created ‎02-08-2017 05:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here is work around, which i have implemented.
For an external table, If you are trying to drop a partition and as-well would like to delete the data. This can be achieved as below.
1. Alter external table as internal table -- by changing the TBL properties as external =false
2. Drop the partitions -- when you drop the partitions, data pertained to the partitions will also be dropped as now this table is managed table
3. Alter back the table as external=True.
By doing this, there more controlled on what we are deleting and drop the partitions rather than using hadoop rm command
Created ‎01-25-2017 01:42 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Reddy
Because it is an external table there is no one-liner to do it. That is probably the whole point of having external tables
So you need to do
ALTER TABLE some.table DROP PARTITION (part="some") PURGE;
and
hdfs dfs -rm -R /path/to/table/basedir
I put the 'PURGE' in there intentionally. It would work for non-external tables, but just not for external tables.
Created ‎01-27-2017 08:38 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Reddy Please don't forget to mark the question as answered if it is answered.
Created ‎02-08-2017 05:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here is work around, which i have implemented.
For an external table, If you are trying to drop a partition and as-well would like to delete the data. This can be achieved as below.
1. Alter external table as internal table -- by changing the TBL properties as external =false
2. Drop the partitions -- when you drop the partitions, data pertained to the partitions will also be dropped as now this table is managed table
3. Alter back the table as external=True.
By doing this, there more controlled on what we are deleting and drop the partitions rather than using hadoop rm command
