question Re: Can falcon move data from hdfs to non-hdfs storage? in Archives of Support Questions (Read Only)

Can falcon move data from hdfs to non-hdfs storage?

sunile_manjee — Fri, 15 Apr 2016 05:01:48 GMT

Can falcon move data from hdfs to non-hdfs storage? I want to move data from hdfs to some external storage area which is non-hdfs. just a simple file system.

Re: Can falcon move data from hdfs to non-hdfs storage?

bvellanki — Fri, 15 Apr 2016 06:14:23 GMT

To add to what @vramachandran said, Falcon also supports replication to Azure ( http://hortonworks.com/hadoop-tutorial/incremental-backup-data-hdp-azure-disaster-recovery-burst-capacity/) and AWS as well.

Re: Can falcon move data from hdfs to non-hdfs storage?

cstanca — Wed, 17 Aug 2016 23:15:46 GMT

@Sunile Manjee

Have you figured-out this? Can Falcon replicate to ext4 file system, for example? None of the responses above addressed your actual question.

Re: Can falcon move data from hdfs to non-hdfs storage?

sunile_manjee — Thu, 18 Aug 2016 04:11:54 GMT

@Constantin Stanca nope.

Re: Can falcon move data from hdfs to non-hdfs storage?

vranganathan — Wed, 07 Sep 2016 10:03:23 GMT

Falcon uses hadoop distributed filesystem abstraction to do the replication -be it s3 or wasb. It essentially uses distcp, so whatever requirements are there for distcp in terms of accessing a filesystem applies to Falcon replication as well