Support Questions

Find answers, ask questions, and share your expertise

How to change the replication factor of data which is already stored in HDFS ?

avatar
Contributor

How to change the replication factor of data which is already stored in HDFS?

1 REPLY 1

avatar
Rising Star

\r\ndfs.replication \r\n4 // To change the replication factor to 4\r\nBlock Replication \r\n \r\nhdfs-site.xml is used to configure HDFS. Changing the dfs.replication property in hdfs-site.xml will change the default replication for all files placed in HDFS.\r\nYou can also change the replication factor on a per-file basis using the Hadoop FS shell.\r\n[training@localhost ~]$ hadoop fs –setrep –w 4 /my/file\r\nAlternatively, you can change the replication factor of all the files under a directory.\r\n[training@localhost ~]$ hadoop fs –setrep –w 4 -R /my/dir\r\n"}" data-sheets-userformat="{"2":769,"3":[null,0],"11":4,"12":0}">The file that is loaded into HDFS which has a default Replication Factor of 3, is set in hdfs-site.xml file. The replication of that particular file would be 3, which means 3 copies of the block exists on the HDFS.

To change the replication factor, open the hdfs-site.xml file. This file is usually found in the conf/ folder of the Hadoop installation directory. Change or add the following property to hdfs-site.xml:
<property>
<name>dfs.replication<name>
<value>4<value> // To change the replication factor to 4
<description>Block Replication<description>
<property>
hdfs-site.xml is used to configure HDFS. Changing the dfs.replication property in hdfs-site.xml will change the default replication for all files placed in HDFS.
You can also change the replication factor on a per-file basis using the Hadoop FS shell.
[training@localhost ~]$ Hadoop fs –setrep –w 4 /my/file
Alternatively, you can change the replication factor of all the files under a directory.
[training@localhost ~]$ Hadoop fs –setrep –w 4 -R /my/dir