Reply
New Contributor
Posts: 5
Registered: ‎02-12-2014

how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

Posts: 416
Topics: 51
Kudos: 88
Solutions: 49
Registered: ‎06-26-2013

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

@putta I'll defer to the HDFS experts, but I don't know if this has been tested and ultimately it will probably come down to whether the hadoop versions on each side are wire compatible.  CDH 4.5 is based on Hadoop 2.0.0, so I'm not sure.

New Contributor
Posts: 5
Registered: ‎02-12-2014

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

Hi clint,

No issues ,it worked for me....we can do it using hftp protocol for the source.......It will sork.

Posts: 1,893
Kudos: 432
Solutions: 302
Registered: ‎07-31-2013

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

Yes, distcp between any two non-wire-compatible Apache Hadoop versions would always require use of either HFTP or WebHDFS protocols instead of HDFS ones.

The goal of HDFS2 is to reach full wire-compatibility eventually down the line, but it isn't fully there yet, last I checked.
New Contributor
Posts: 5
Registered: ‎02-12-2014

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

Thanks :smileyhappy:

Announcements