Id HDFS files replication with Delta possible using Falcon UI

[#1]

Hi,

I have successfully performed Data replication for HDFS files using Falcon. Now, I would like to perform HDFS file replication with Delta . The HDFS files shouldn't be overwritten during Data replication. Instead, HDFS files would get updated.

[#2 · Accepted solution]

@Vishakha Agarwal

As of HDP 2.5 (Falcon 0.10), Falcon supports snapshot-based replication. This will only replicate deltas. For details on the JIRA and design document/explanation take a look at the below links:

https://issues.apache.org/jira/browse/FALCON-1861

https://issues.apache.org/jira/secure/attachment/12800139/HDFSSnapshotsBasedReplication.pdf

Having said that, please be advised that Falcon has been deprecated starting HDP 2.6. Expect to see a replacement tool in a future version.

As always, if you find this post helpful, don't forget to "accept" answer.