question Re: Falcon HDFS Mirror deletions in Archives of Support Questions (Read Only)

Falcon HDFS Mirror deletions

jknulst — Thu, 20 Oct 2016 05:10:49 GMT

Hi,

I set up a Falcon HDFS mirror. Inserts and file updates in the Src are handled just fine. Only deletions from the Src are not reflected in the Tgt. Is that a config thing or just not supported?

Re: Falcon HDFS Mirror deletions

sramesh — Thu, 20 Oct 2016 05:17:39 GMT

@Jasper

Pass removeDeletedFiles property in the request. This maps to -delete option in DistCP. Thanks!

Re: Falcon HDFS Mirror deletions

jknulst — Thu, 20 Oct 2016 05:25:22 GMT

@Sowmya Ramesh is there a hook for that in the Falcon WebUI? Is it that TAG > Value thing maybe

Re: Falcon HDFS Mirror deletions

sramesh — Thu, 20 Oct 2016 05:34:28 GMT

Looks like this is not supported in UI. You can do a it through REST API or command line. Refer https://falcon.apache.org/restapi/ExtensionSubmitAndSchedule.html

or update existing mirror jobs using https://falcon.apache.org/restapi/ExtensionUpdate.html

Add "removeDeletedFiles=true" as POSt parameter. Thanks!