I have successfully performed Data replication for HDFS files using Falcon. Now, I would like to perform HDFS file replication with Delta . The HDFS files shouldn't be overwritten during Data replication. Instead, HDFS files would get updated.
As of HDP 2.5 (Falcon 0.10), Falcon supports snapshot-based replication. This will only replicate deltas. For details on the JIRA and design document/explanation take a look at the below links:
Having said that, please be advised that Falcon has been deprecated starting HDP 2.6. Expect to see a replacement tool in a future version.
As always, if you find this post helpful, don't forget to "accept" answer.
View solution in original post