Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Which nodes are participating in Under Replication in HDFS

Solved Go to solution
Highlighted

Which nodes are participating in Under Replication in HDFS

New Contributor

Hi,

I had 1 millions of files under replicated in my HDFS cluster for a testing scenario, It replicated successfully but I want to know that Which nodes are participating on this operation ?

Like What is the role of namenode and datanode in this under-replication process ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Which nodes are participating in Under Replication in HDFS

The ALL are; especially when talking about so many files that were under-replicated. Ultimately, the NN is the one who determines if a file is under-replicated. It is then the NN's job to notify one of the DNs that has a good copy of one of the blocks' replicas to copy it to another DN. NN isn't going to do any of the actual movement of bits -- it will just coordinate the whole effort. Hope this helps!

8766-repl2.png

2 REPLIES 2

Re: Which nodes are participating in Under Replication in HDFS

The ALL are; especially when talking about so many files that were under-replicated. Ultimately, the NN is the one who determines if a file is under-replicated. It is then the NN's job to notify one of the DNs that has a good copy of one of the blocks' replicas to copy it to another DN. NN isn't going to do any of the actual movement of bits -- it will just coordinate the whole effort. Hope this helps!

8766-repl2.png

Re: Which nodes are participating in Under Replication in HDFS

Don't have an account?
Coming from Hortonworks? Activate your account here