Support Questions
Find answers, ask questions, and share your expertise

Datnode is not alive tough datanode is up

Datnode is not alive tough datanode is up


We have 3 datanodes in and only 2 datanodes are alive tough 3 datanodes are up ,amoung 3 datanodes 2 datanodes are in default rack and 1 datanode is in specified rack ,datanode in specified rack is not going live


tied refreshing and restarting of entire hdfs ,nothing helped 

could see below error in logs 


2020-09-16 13:13:24,006 ERROR Error: can't add leaf node /AZ/EUS2/ambari/0/ at depth 6 to topology:Number of rcks: 1 Expected number of leaves:2 /default-rack/ /default-rack/
2020-09-16 13:13:24,007 WARN IPC Server handler 182 on 8020, call Call#419 Retry#0 org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.registerDatanode from$InvalidTopologyException: Failed to add /AZ/EUS2/ambari//0/ You cannot have a rack and a non-rack node at the same level of the network topology. at at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode( at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.registerDatanode( at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode( at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode( at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode( at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod( at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ at org.apache.hadoop.ipc.RPC$ at org.apache.hadoop.ipc.Server$ at org.apache.hadoop.ipc.Server$ at Method) at at at org.apache.hadoop.ipc.Server$