question Re: Which nodes are participating in Under Replication in HDFS in Archives of Support Questions (Read Only)

Which nodes are participating in Under Replication in HDFS

viruvekariya — Fri, 21 Oct 2016 14:58:18 GMT

Hi,

I had 1 millions of files under replicated in my HDFS cluster for a testing scenario, It replicated successfully but I want to know that Which nodes are participating on this operation ?

Like What is the role of namenode and datanode in this under-replication process ?

Re: Which nodes are participating in Under Replication in HDFS

LesterMartin — Mon, 19 Aug 2019 08:08:34 GMT

The ALL are; especially when talking about so many files that were under-replicated. Ultimately, the NN is the one who determines if a file is under-replicated. It is then the NN's job to notify one of the DNs that has a good copy of one of the blocks' replicas to copy it to another DN. NN isn't going to do any of the actual movement of bits -- it will just coordinate the whole effort. Hope this helps!

Re: Which nodes are participating in Under Replication in HDFS

LesterMartin — Sat, 22 Oct 2016 00:26:19 GMT

More details can be viewed from the "source" at http://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html