Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to change the replication factor of data which is already stored in HDFS ?

How to change the replication factor of data which is already stored in HDFS ?

How to change the replication factor of data which is already stored in HDFS?

1 REPLY 1
Highlighted

Re: How to change the replication factor of data which is already stored in HDFS ?

\r\ndfs.replication \r\n4 // To change the replication factor to 4\r\nBlock Replication \r\n \r\nhdfs-site.xml is used to configure HDFS. Changing the dfs.replication property in hdfs-site.xml will change the default replication for all files placed in HDFS.\r\nYou can also change the replication factor on a per-file basis using the Hadoop FS shell.\r\n[training@localhost ~]$ hadoop fs –setrep –w 4 /my/file\r\nAlternatively, you can change the replication factor of all the files under a directory.\r\n[training@localhost ~]$ hadoop fs –setrep –w 4 -R /my/dir\r\n"}" data-sheets-userformat="{"2":769,"3":[null,0],"11":4,"12":0}">The file that is loaded into HDFS which has a default Replication Factor of 3, is set in hdfs-site.xml file. The replication of that particular file would be 3, which means 3 copies of the block exists on the HDFS.

To change the replication factor, open the hdfs-site.xml file. This file is usually found in the conf/ folder of the Hadoop installation directory. Change or add the following property to hdfs-site.xml:
<property>
<name>dfs.replication<name>
<value>4<value> // To change the replication factor to 4
<description>Block Replication<description>
<property>
hdfs-site.xml is used to configure HDFS. Changing the dfs.replication property in hdfs-site.xml will change the default replication for all files placed in HDFS.
You can also change the replication factor on a per-file basis using the Hadoop FS shell.
[training@localhost ~]$ Hadoop fs –setrep –w 4 /my/file
Alternatively, you can change the replication factor of all the files under a directory.
[training@localhost ~]$ Hadoop fs –setrep –w 4 -R /my/dir

Don't have an account?
Coming from Hortonworks? Activate your account here