Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Id HDFS files replication with Delta possible using Falcon UI

Solved Go to solution
Highlighted

Id HDFS files replication with Delta possible using Falcon UI

Explorer

Hi,

I have successfully performed Data replication for HDFS files using Falcon. Now, I would like to perform HDFS file replication with Delta . The HDFS files shouldn't be overwritten during Data replication. Instead, HDFS files would get updated.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Id HDFS files replication with Delta possible using Falcon UI

@Vishakha Agarwal

As of HDP 2.5 (Falcon 0.10), Falcon supports snapshot-based replication. This will only replicate deltas. For details on the JIRA and design document/explanation take a look at the below links:

https://issues.apache.org/jira/browse/FALCON-1861

https://issues.apache.org/jira/secure/attachment/12800139/HDFSSnapshotsBasedReplication.pdf

Having said that, please be advised that Falcon has been deprecated starting HDP 2.6. Expect to see a replacement tool in a future version.

As always, if you find this post helpful, don't forget to "accept" answer.

View solution in original post

1 REPLY 1
Highlighted

Re: Id HDFS files replication with Delta possible using Falcon UI

@Vishakha Agarwal

As of HDP 2.5 (Falcon 0.10), Falcon supports snapshot-based replication. This will only replicate deltas. For details on the JIRA and design document/explanation take a look at the below links:

https://issues.apache.org/jira/browse/FALCON-1861

https://issues.apache.org/jira/secure/attachment/12800139/HDFSSnapshotsBasedReplication.pdf

Having said that, please be advised that Falcon has been deprecated starting HDP 2.6. Expect to see a replacement tool in a future version.

As always, if you find this post helpful, don't forget to "accept" answer.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here