Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Replication factor not changed in HDFS from hdfs-site.xml

Solved Go to solution

Replication factor not changed in HDFS from hdfs-site.xml

New Contributor

I was facing failed to replace bad datanode error while appending new data to file and the work around was to set dfs.replication to less than 3 , so I set it to 1 just to test it. But I still got the same error. I looked at the hadoop web interface and surprisingly the replication factor was still 3 . but when I did hdfs dfs -setrep 1 <file_name> the replication is set to 1 and I could append to file. Why is this happening? Can I not set default replication factor ? I tried formating namenode still no change.

Here's my hdfs-site.xml

<configuration>
<property>
  <name>dfs.replication</name>
  <value>1</value>
</property>

<property>
  <name>dfs.permissions</name>
  <value>false</value>
</property>

  <property> 
    <name>dfs.support.append</name>
    <value>true</value> 
</property>
</configuration>

I tried to follow steps from this question, still my replication factor is 3. I am running hadoop in single node cluster.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Replication factor not changed in HDFS from hdfs-site.xml

@Saurab Dahal - As in the other answers the change in hdfs-site.xml will only affect the newly created files with the new replication factor.

If you want to change replication factor for already existing files. You can run the hdfs cli commands like below:

hdfs dfs -setrep -w 1 /myfile
hdfs dfs -setrep -w 1 /mydir

Hope this helps !

4 REPLIES 4
Highlighted

Re: Replication factor not changed in HDFS from hdfs-site.xml

That will only set it for newly created directories. Using the HDFS client, set the replication factor for the directory to the new value.

Re: Replication factor not changed in HDFS from hdfs-site.xml

Contributor

@Saurab Dahal The replication factor is already set to 3 for the file you are trying to append data to. Even if the config value is changed, it gets into effect for new files. Please check the replication factor for a file created after the config value change.

Re: Replication factor not changed in HDFS from hdfs-site.xml

New Contributor

Hi Saurab, Are u able to see the changed value in the logs , while restarting the namenode. you can check in the namenode startup logs like.

grep -ri dfs.replication *

Re: Replication factor not changed in HDFS from hdfs-site.xml

@Saurab Dahal - As in the other answers the change in hdfs-site.xml will only affect the newly created files with the new replication factor.

If you want to change replication factor for already existing files. You can run the hdfs cli commands like below:

hdfs dfs -setrep -w 1 /myfile
hdfs dfs -setrep -w 1 /mydir

Hope this helps !