Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS trash not clearing the files

HDFS trash not clearing the files

Contributor

Hi,

 

We have enbaled trash in our hadoop cluster and trash interval is 1 day.

 

But still we see lakhs of files in trash.

 

Can you please suggest ?

 

Thanks,

Priya

5 REPLIES 5

Re: HDFS trash not clearing the files

Master Guru
What version of Cloudera Manager (CM) are you using? What is the value of CM - HDFS - Configuration - "Filesystem Trash Checkpoint Interval"? It should ideally be 1 hour, so any deleted file can get deleted in ~1d +1h after being moved to trash.

Re: HDFS trash not clearing the files

Contributor
@Harsh J,

We are using 5.9.2 version of cloudera manager.
Value for "Filesystem Trash Checkpoint Interval" is 1 day.

Re: HDFS trash not clearing the files

Contributor
@Harsh J,

We are using 5.9.2 version of cloudera manager.
I don't see Filesystem Trash Checkpoint Interval parameter in Cloudera manager. Value for Filesystem Trash Interval is 1 day.

Please suggest.

Thanks,
Priya

Re: HDFS trash not clearing the files

Master Guru
The property control field was introduced with a default of 1 hour in Cloudera Manager 5.11.0 onwards. See https://www.cloudera.com/documentation/enterprise/release-notes/topics/cm_rn_new_changed_features.ht...

In your current configuration, trash deletion may take upto 3 days, but should certainly happen.

If you cannot currently upgrade your Cloudera Manager, you can add the below to your HDFS - Configuration - "NameNode Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml" field:

Name: fs.trash.checkpoint.interval
Value: 60

Or if in XML form:

<property>
<name>fs.trash.checkpoint.interval</name><value>60</value>
</property>

Re: HDFS trash not clearing the files

Contributor
@Harsh J,

Thanks for valuable inputs. So after setting checkpoint value to 1 hour, files will be deleted after 1 day +1 hour.

Thanks,
Priya
Don't have an account?
Coming from Hortonworks? Activate your account here