HDFS trash not clearing the files [ Edited ]

[cdhhadoop]

Hi,

We have enbaled trash in our hadoop cluster and trash interval is 1 day.

But still we see lakhs of files in trash.

Can you please suggest ?

Thanks,

Priya

[Harsh J Cloudera Employee]

What version of Cloudera Manager (CM) are you using? What is the value of CM - HDFS - Configuration - "Filesystem Trash Checkpoint Interval"? It should ideally be 1 hour, so any deleted file can get deleted in ~1d +1h after being moved to trash.

[cdhhadoop]

@Harsh J,

We are using 5.9.2 version of cloudera manager.
Value for "Filesystem Trash Checkpoint Interval" is 1 day.

[cdhhadoop]

@Harsh J,

We are using 5.9.2 version of cloudera manager.
I don't see Filesystem Trash Checkpoint Interval parameter in Cloudera manager. Value for Filesystem Trash Interval is 1 day.

Please suggest.

Thanks,
Priya

[Harsh J Cloudera Employee]

The property control field was introduced with a default of 1 hour in Cloudera Manager 5.11.0 onwards. See https://www.cloudera.com/documentation/enterprise/release-notes/topics/cm_rn_new_changed_features.ht...

In your current configuration, trash deletion may take upto 3 days, but should certainly happen.

If you cannot currently upgrade your Cloudera Manager, you can add the below to your HDFS - Configuration - "NameNode Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml" field:

Name: fs.trash.checkpoint.interval
Value: 60

Or if in XML form:

<property>
<name>fs.trash.checkpoint.interval</name><value>60</value>
</property>

[cdhhadoop]

@Harsh J,

Thanks for valuable inputs. So after setting checkpoint value to 1 hour, files will be deleted after 1 day +1 hour.

Thanks,
Priya