Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to change the replication factor of data which is already stored in HDFS ?

avatar
Contributor

How to change the replication factor of data which is already stored in HDFS?

1 REPLY 1

avatar
Rising Star

\r\ndfs.replication \r\n4 // To change the replication factor to 4\r\nBlock Replication \r\n \r\nhdfs-site.xml is used to configure HDFS. Changing the dfs.replication property in hdfs-site.xml will change the default replication for all files placed in HDFS.\r\nYou can also change the replication factor on a per-file basis using the Hadoop FS shell.\r\n[training@localhost ~]$ hadoop fs –setrep –w 4 /my/file\r\nAlternatively, you can change the replication factor of all the files under a directory.\r\n[training@localhost ~]$ hadoop fs –setrep –w 4 -R /my/dir\r\n"}" data-sheets-userformat="{"2":769,"3":[null,0],"11":4,"12":0}">The file that is loaded into HDFS which has a default Replication Factor of 3, is set in hdfs-site.xml file. The replication of that particular file would be 3, which means 3 copies of the block exists on the HDFS.

To change the replication factor, open the hdfs-site.xml file. This file is usually found in the conf/ folder of the Hadoop installation directory. Change or add the following property to hdfs-site.xml:
<property>
<name>dfs.replication<name>
<value>4<value> // To change the replication factor to 4
<description>Block Replication<description>
<property>
hdfs-site.xml is used to configure HDFS. Changing the dfs.replication property in hdfs-site.xml will change the default replication for all files placed in HDFS.
You can also change the replication factor on a per-file basis using the Hadoop FS shell.
[training@localhost ~]$ Hadoop fs –setrep –w 4 /my/file
Alternatively, you can change the replication factor of all the files under a directory.
[training@localhost ~]$ Hadoop fs –setrep –w 4 -R /my/dir