Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Replication factor not changed in HDFS from hdfs-site.xml

avatar
Contributor

I was facing failed to replace bad datanode error while appending new data to file and the work around was to set dfs.replication to less than 3 , so I set it to 1 just to test it. But I still got the same error. I looked at the hadoop web interface and surprisingly the replication factor was still 3 . but when I did hdfs dfs -setrep 1 <file_name> the replication is set to 1 and I could append to file. Why is this happening? Can I not set default replication factor ? I tried formating namenode still no change.

Here's my hdfs-site.xml

<configuration>
<property>
  <name>dfs.replication</name>
  <value>1</value>
</property>

<property>
  <name>dfs.permissions</name>
  <value>false</value>
</property>

  <property> 
    <name>dfs.support.append</name>
    <value>true</value> 
</property>
</configuration>

I tried to follow steps from this question, still my replication factor is 3. I am running hadoop in single node cluster.

1 ACCEPTED SOLUTION

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar

That will only set it for newly created directories. Using the HDFS client, set the replication factor for the directory to the new value.

avatar
Rising Star

@Saurab Dahal The replication factor is already set to 3 for the file you are trying to append data to. Even if the config value is changed, it gets into effect for new files. Please check the replication factor for a file created after the config value change.

avatar
Contributor

Hi Saurab, Are u able to see the changed value in the logs , while restarting the namenode. you can check in the namenode startup logs like.

grep -ri dfs.replication *

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login