Created on 06-13-2017 11:56 AM - edited 09-16-2022 04:44 AM
Is it possible to use different Trash interval for hdfs directories. Eg Directory "Market" for trash interval 7days and directory search for "14" days?
Created 06-13-2017 12:00 PM
Created 06-13-2017 02:37 PM
By default CM provides a common setting for fs.trash.interval like 1 day, 2 days, etc
But you can achieve your needs by writting a customized shell script and schedule it
a. to move trash data to a back-up folder every day
b. delete 7 days old data automatically
Created 06-22-2017 02:00 PM
Created 06-22-2017 02:05 PM
The HDFS trash policy is pluggable. The default (and the only one supported by Cloudera) is TrashPolicyDefault (org.apache.hadoop.fs.TrashPolicyDefault). But in theory you could write your own trash policy and configure NameNode to use your trash policy.
If you are interested to see it happen, file a feature request jira at Apache Hadoop https://issues.apache.org/jira/projects/HDFS and see if someone would pick it up.
Created 06-22-2017 03:32 PM