Support Questions
Find answers, ask questions, and share your expertise

Falcon HDFS Mirror deletions

Solved Go to solution
Highlighted

Falcon HDFS Mirror deletions

Super Collaborator

Hi,

I set up a Falcon HDFS mirror. Inserts and file updates in the Src are handled just fine. Only deletions from the Src are not reflected in the Tgt. Is that a config thing or just not supported?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Falcon HDFS Mirror deletions

@Jasper

Pass removeDeletedFiles property in the request. This maps to -delete option in DistCP. Thanks!

View solution in original post

3 REPLIES 3
Highlighted

Re: Falcon HDFS Mirror deletions

@Jasper

Pass removeDeletedFiles property in the request. This maps to -delete option in DistCP. Thanks!

View solution in original post

Highlighted

Re: Falcon HDFS Mirror deletions

Super Collaborator

@Sowmya Ramesh is there a hook for that in the Falcon WebUI? Is it that TAG > Value thing maybe

Highlighted

Re: Falcon HDFS Mirror deletions

Looks like this is not supported in UI. You can do a it through REST API or command line. Refer https://falcon.apache.org/restapi/ExtensionSubmitAndSchedule.html

or update existing mirror jobs using https://falcon.apache.org/restapi/ExtensionUpdate.html

Add "removeDeletedFiles=true" as POSt parameter. Thanks!

Don't have an account?