Reply
rio
Explorer
Posts: 48
Registered: ‎04-18-2014

Transferring data between CDH

[ Edited ]

Hi,

 

How do I transfer data between servers in 2 different data centers both running CDH4?

 

Thanks!

Posts: 1,886
Kudos: 425
Solutions: 300
Registered: ‎07-31-2013

Re: Transferring data between CDH 4 and CDH 4

Could you clarify - what kind of data? HDFS files? (DistCp) Log/Event
files? (Flume) Or something else?

rio
Explorer
Posts: 48
Registered: ‎04-18-2014

Re: Transferring data between CDH 4 and CDH 4

Both clusters are running cdh4.3.1.

 

I want to transfer data from one HDFS to another HDFS.

 

What are all possible ways?

 

Thank you!

Highlighted
Posts: 1,886
Kudos: 425
Solutions: 300
Registered: ‎07-31-2013

Re: Transferring data between CDH 4 and CDH 4

There are a few ways:

Simple copies, run on dest.: hadoop fs -cp hdfs://source-nn/path
hdfs://dest-nn/path

Distributed (large) copies, run on dest.: hadoop
distcp hdfs://source-nn/path hdfs://dest-nn/path

Or if you have a Cloudera Enterprise license, use BDR:
http://www.cloudera.com/content/cloudera/en/resources/library/training/cloudera-enterprise-bdr-overv...

rio
Explorer
Posts: 48
Registered: ‎04-18-2014

Re: Transferring data between CDH 4 and CDH 4

[ Edited ]

How do I transfer data between below clusters:

 

1. CDH4 to CDH5 

 

2. CDH5 to CDH4

 

3. CDH5 to CDH5

 

 

Posts: 1,886
Kudos: 425
Solutions: 300
Registered: ‎07-31-2013

Re: Transferring data between CDH 4 and CDH 4

Could you clarify your goal here? The answer is still DistCp. Have you
looked at our documentation yet, and specifically have you looked at our
DistCp matrix page yet?
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_admin_distcp_data_c...

Announcements
New solutions