if it is the case of 3 node cluster we have 1 node for namenode 2 nodes for datanodes how replication works . we have default 3 replicas? i am confused

[#1]

@gajulapalli naveen

If replication factor is 3 (which is default replication factor) But you have only two datanodes then in that case the Data blocks will be replicated to 2 DataNodes and you might see many "Under Replicated Blocks" because there is not 3rd DataNode.

As you have only 3 node cluster, So either you should install DataNode on the 3rd Host as well (where the NameNode is installed)

OR

you should change the "dfs.replication" property value from default 3 to 2 inside your "hdfs-site,xml"

[#2]

thank you @Jay Kumar SenSharma ... as u said name node will act as datanode? but name node only for metadata right? can we assign namenode as datanode?

[#3 · Accepted solution]

@gajulapalli naveen

I never said "name node will act as datanode". What i meant to say is that ... On the same Physical Host you can install DataNode and NameNode both the processes.

NameNode stored the metadata of the HDFS and the DataNdoe actually stores the block data.

.

What i mean to say is that inside your NameNode host you can also install DataNode without any issue. You should see both NameNode & DataNode process running on the same host at the same time as two separate JVM process.

# ps -ef | grep DataNode
# ps -ef | grep NameNode

.

Example Cluster setup for 3 node cluster environment

--------------------------------------------------------------------------------

For Non HA Setup (3 node cluster)

Host1 --> NameNode  and DataNode
Host2 --> NameNode(Secondary) DataNode
Host3 --> DataNode

.

For NameNode HA Setup (3 node cluster)

Host1 --> NameNode (Active)  and DataNode
Host2 --> NameNode (StandBy)  and DataNode
Host3 --> DataNode

.

.

[#4]

@gajulapalli naveen

If this answers your query then please mark this thread as answered by clicking on the "Accept" Button on the correct answer. That way other HCC users can quickly find the solution when they encounter the same issue/query.

[#5]

@Jay Kumar SenSharma Thank you for information