Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

New Contributor

how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

4 REPLIES 4

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

Master Collaborator

@putta I'll defer to the HDFS experts, but I don't know if this has been tested and ultimately it will probably come down to whether the hadoop versions on each side are wire compatible.  CDH 4.5 is based on Hadoop 2.0.0, so I'm not sure.

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

New Contributor

Hi clint,

No issues ,it worked for me....we can do it using hftp protocol for the source.......It will sork.

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

Master Guru
Yes, distcp between any two non-wire-compatible Apache Hadoop versions would always require use of either HFTP or WebHDFS protocols instead of HDFS ones.

The goal of HDFS2 is to reach full wire-compatibility eventually down the line, but it isn't fully there yet, last I checked.
Highlighted

Re: how distcp works for data transfer between cloudera CDH4.5 and Hortonworks (hadoop 2.2)?

New Contributor

Thanks :smileyhappy:

Don't have an account?
Coming from Hortonworks? Activate your account here