question Move data inter datanodes in Archives of Support Questions (Read Only)

Move data inter datanodes

rodrigorondena1 — Mon, 22 May 2017 21:51:34 GMT

Hello Guys,

My question is regarding to move data inter clusters, if I need to migrate the data from minor version of HDP to a higher version.

If I update the version of HDFS in old datanodes and include those datanodes in new version, the HDFS will are able to see the files that were allocated in old version ? Do I need to execute the rebalance if it works ?

or the best option is using the DISTCP and move data inter cluster versions ?

Rgs,

R.R.

Re: Move data inter datanodes

mqureshi — Tue, 23 May 2017 03:26:09 GMT

@Rodrigo Rondena

So you want to distcp data from one cluster to another cluster and the two clusters are on different versions. Is that right? Are they on same major version, for example, 2.x and 2.y? In this case you just simply use hdfs protocol. To distcp between two different major versions, use webhdfs protocol. Check the following link:

https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html#Copying_Between_Versions_of_HDFS