Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Migrate data from one cluster to another cluster

Migrate data from one cluster to another cluster

Explorer

Dear Experts, we have one use case where we have to migrate data (peta bytes) from one cluster to another cluster.

What is strategy and tool I sould consider to migrate ~10 pb of data from one cluster to another cluster.

 

Many Thanks  in advance for your help!

4 REPLIES 4

Re: Migrate data from one cluster to another cluster

Community Manager

Hi @xBigDatax,

 

 

What data are we talking about here? HDFS, Hive, HBase, Impala, Search etc? I assume you are not using Cloudera Manager?

 

Thanks,

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Re: Migrate data from one cluster to another cluster

Explorer

Yes, I am talking about  HDFS, Hive, HBase, Impala, Search etc and also cloudera Manager too from one Cloudera Cluster to Another cloudera cluster

data size in petabytes. Hope  this clarifies

Thanks

Re: Migrate data from one cluster to another cluster

Community Manager

Hi @xBigDatax,

 

You can take a look at BDR feature (which should handle HDFS, Hive, Impala) since you are using Cloudera Manager:

https://www.cloudera.com/documentation/enterprise/6/latest/topics/cm_bdr_about.html

https://blog.cloudera.com/blog/2016/08/considerations-for-production-environments-running-cloudera-b...

 

For search, you can take a look at this blog:

https://blog.cloudera.com/blog/2017/05/how-to-backup-and-disaster-recovery-for-apache-solr-part-i/

and public doc:

https://www.cloudera.com/documentation/enterprise/6/latest/topics/search_backup_restore.html

 

For HBase, you can take a look at this:

https://www.cloudera.com/documentation/enterprise/6/latest/topics/cdh_bdr_hbase_replication.html

 

You can also explore the distcp tool which is a general utility for copying large data sets between distributed filesystems within and across clusters.

https://www.cloudera.com/documentation/enterprise/6/6.1/topics/cdh_admin_distcp_cdh.html

 

Thanks and hope this helps,

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Highlighted

Re: Migrate data from one cluster to another cluster

Explorer

Please suggest me for intial load of TBs and/or PBs of data from Proudcution cluster to DR cluster. Do you suggest to migrate/replicate data using Cloudera BDR?

 

Thanks