Reply
Contributor
Posts: 28
Registered: ‎11-27-2017

Drop partition remove data but not HDFS folder

Hi,

 

We have CDH 5.15 and when we drop partition in Hive it removes the partition and its data but the folder of the partition remains empty and is not removed.

 

How can I change this behavior?

 

Thanks

Posts: 49
Topics: 0
Kudos: 13
Solutions: 5
Registered: ‎07-30-2018

Re: Drop partition remove data but not HDFS folder

Hi,

If it is a External table we need to delete the directory manually.

ALTER TABLE table_name DROP [IF EXISTS] PARTITION <>;
hadoop fs -rm -r <partition file path>

Thanks
Jerry
Contributor
Posts: 28
Registered: ‎11-27-2017

Re: Drop partition remove data but not HDFS folder

Hi,

 

It's an internal table with 4 levels of partitions,

But still, when I remove the partition the data and metadata is deleted but not the folder.

 

Thanks

Cloudera Employee
Posts: 824
Registered: ‎03-23-2015

Re: Drop partition remove data but not HDFS folder

Hi,

I assume that you work on the managed table instead of external table? This could be because of lack of permissions from the user who tried to run the DROP command to remove the underlining HDFS path. Check HMS server log to see if you can find any error messages.