question How to reset “Reserved Space for Replicas” without restart Hadoop services in Support Questions

How to reset “Reserved Space for Replicas” without restart Hadoop services

Zholaman — Mon, 01 Jun 2020 19:10:31 GMT

How to find why "Reserved Space for Replicas" constantly increasing and how to control/set a space size allocated for the "Reserved Space for Replicas" in Hadoop?

We found that the "Reserved Space for Replicas" not relate to "Non DFS" space and it can increase constantly up to rebooting Data Node-s. We didn't find 😞 how to limit space allocated for the "Reserved Space for Replicas" 😞

We thought that dfs.datanode.du.reserved can control "Reserved Space for Replicas" but it is not!

Re: How to reset “Reserved Space for Replicas” without restart Hadoop services

Zholaman — Tue, 02 Jun 2020 05:21:10 GMT

We found that the "Reserved Space for Replicas" ~~not~~ relate to "Non DFS" space

Re: How to reset “Reserved Space for Replicas” without restart Hadoop services

Zholaman — Wed, 03 Jun 2020 05:29:02 GMT

For people who faced with that type of problem. First of all you should understand the nature of the problem. To do that please read the description of the following issues:

https://issues.apache.org/jira/browse/HDFS-8072
https://issues.apache.org/jira/browse/HDFS-9530

The following links will be useful to understand what block replica is:

https://blog.cloudera.com/understanding-hdfs-recovery-processes-part-1/
https://blog.cloudera.com/understanding-hdfs-recovery-processes-part-2/

Solutions

1. Find the wrong software which often broke connection with Hadoop during the write or append processes

2. Try to change replication policy (risky)

3. Update Hadoop up to late version

You can't reset “Reserved Space for Replicas” without restart Hadoop services!