Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Id HDFS files replication with Delta possible using Falcon UI

Explorer

Hi,

I have successfully performed Data replication for HDFS files using Falcon. Now, I would like to perform HDFS file replication with Delta . The HDFS files shouldn't be overwritten during Data replication. Instead, HDFS files would get updated.

1 ACCEPTED SOLUTION

@Vishakha Agarwal

As of HDP 2.5 (Falcon 0.10), Falcon supports snapshot-based replication. This will only replicate deltas. For details on the JIRA and design document/explanation take a look at the below links:

https://issues.apache.org/jira/browse/FALCON-1861

https://issues.apache.org/jira/secure/attachment/12800139/HDFSSnapshotsBasedReplication.pdf

Having said that, please be advised that Falcon has been deprecated starting HDP 2.6. Expect to see a replacement tool in a future version.

As always, if you find this post helpful, don't forget to "accept" answer.

View solution in original post

1 REPLY 1

@Vishakha Agarwal

As of HDP 2.5 (Falcon 0.10), Falcon supports snapshot-based replication. This will only replicate deltas. For details on the JIRA and design document/explanation take a look at the below links:

https://issues.apache.org/jira/browse/FALCON-1861

https://issues.apache.org/jira/secure/attachment/12800139/HDFSSnapshotsBasedReplication.pdf

Having said that, please be advised that Falcon has been deprecated starting HDP 2.6. Expect to see a replacement tool in a future version.

As always, if you find this post helpful, don't forget to "accept" answer.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.