Id HDFS files replication with Delta possible using Falcon UI

vishakhaa9 — Thu, 01 Jun 2017 18:45:30 GMT

Hi,

I have successfully performed Data replication for HDFS files using Falcon. Now, I would like to perform HDFS file replication with Delta . The HDFS files shouldn't be overwritten during Data replication. Instead, HDFS files would get updated.

Re: Id HDFS files replication with Delta possible using Falcon UI

egarelnabi — Thu, 01 Jun 2017 20:52:42 GMT

@Vishakha Agarwal

As of HDP 2.5 (Falcon 0.10), Falcon supports snapshot-based replication. This will only replicate deltas. For details on the JIRA and design document/explanation take a look at the below links:

https://issues.apache.org/jira/browse/FALCON-1861

https://issues.apache.org/jira/secure/attachment/12800139/HDFSSnapshotsBasedReplication.pdf

Having said that, please be advised that Falcon has been deprecated starting HDP 2.6. Expect to see a replacement tool in a future version.

As always, if you find this post helpful, don't forget to "accept" answer.

question Id HDFS files replication with Delta possible using Falcon UI in Archives of Support Questions (Read Only)

Id HDFS files replication with Delta possible using Falcon UI

Re: Id HDFS files replication with Delta possible using Falcon UI