Reply
Contributor
Posts: 47
Registered: ‎12-28-2015

HDFS Trash interval

Is it possible to use different Trash interval for hdfs directories. Eg Directory "Market" for trash interval 7days and directory search for "14" days?

 

 

Posts: 642
Topics: 3
Kudos: 120
Solutions: 67
Registered: ‎08-16-2016

Re: HDFS Trash interval

No. When deleting a file or folder it is access and then determines whether to send it to trash or not. I also believe there is a cleaner process that then checks it to decide when to clear items out of the trash.
Posts: 519
Topics: 14
Kudos: 92
Solutions: 45
Registered: ‎09-02-2016

Re: HDFS Trash interval

@naveen1

 

By default CM provides a common setting for fs.trash.interval like 1 day, 2 days, etc

 

But you can achieve your needs by writting a customized shell script and schedule it 

a. to move trash data to a back-up folder every day

b. delete 7 days old data automatically

 

 

Contributor
Posts: 47
Registered: ‎12-28-2015

Re: HDFS Trash interval

This seems to be a overhead and I was seeing if hdfs could handle this request. Thanks for all replies.
Cloudera Employee
Posts: 47
Registered: ‎08-16-2016

Re: HDFS Trash interval

The HDFS trash policy is pluggable. The default (and the only one supported by Cloudera) is TrashPolicyDefault (org.apache.hadoop.fs.TrashPolicyDefault). But in theory you could write your own trash policy and configure NameNode to use your trash policy.

 

If you are interested to see it happen, file a feature request jira at Apache Hadoop https://issues.apache.org/jira/projects/HDFS and see if someone would pick it up.

Highlighted
Contributor
Posts: 47
Registered: ‎12-28-2015

Re: HDFS Trash interval

I have opened a Jira for this request. Thanks Wei
Announcements