Support Questions
Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

How to do I copy data from one HDFS to another HDFS?


I have two HDFS setup and want to copy (not migrate or move) some tables from HDFS1 to HDFS2. How to do I copy data from one HDFS to another HDFS? Other than using sqoop or discp options.



Your only option outside of distcp and recreating the tables on the other cluster is to use Falcon.

It still uses distcp in the background though, but that is transparent to the user.

Please be advised that starting HDP 2.6, Falcon has been deprecated and will be completely removed from the stack starting HDP 3

As always, if you find this post helpful, don't forget to "accept" answer.


There's currently no substitute to Falcon or distcp within the platform. Expect a solution in the near future that will replace the deprecated Falcon.

Having said that, I would suggest you take the distcp and recreating/copying the Hive DDL/tables route rather than investing effort into setting up Falcon.


Thank you Eyad, we are using HDP 2.6, do we have any other option other than Falcon. Since Falcon has been deprecated with HDP 2.6.


You could also try taking a HDFS snapshot:

You can setup a cron job that takes the snapshot and does the copy on a regular basis.