Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

The problem with the entering datanode to maintenance state

avatar
New Contributor

There is a problem with entering a host to the maintenance state using cloudera manager. The problem is related to replication of some blocks. The most files in hdfs have a factor replication of 3, but a few of them have changed at 2. When maintenance is starting the first of the blocks with replication factor of 2 trying to replicate to another DataNode and can't do that. The value dfs.namenode.maintenance.replication.min is 1. In the NameNode log file there is a message (it repeats):


INFO BlockStateChange: Block: blk_2355920052_1283317658, Expected Replicas: 2, live replicas: 0, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 0, maintenance replicas: 1, live entering maintenance replicas: 1, excess replicas: 1, Is Open File: false, Datanodes having this block: ... ... , Current Datanode: ..., Is current datanode decommissioning: false, Is current datanode entering maintenance: true
 
and simultaneously the fsck command by that block (hdfs fsck / -blockId blk_2355920052) shows:

No. of Expected Replicas: 2
No. of live Replica: 1
No. of excess Replica: 0
No. of stale Replica: 0
No. of decommissioned Replica: 0
No. of decommissioning Replica: 0
No. of corrupt Replica: 0
Block replica on dnanode ... is HEALTHY
Block replica on dnanode ... is HEALTHY
 
On the one hand NameNode shows that  live replicas: 0 and  excess replicas: 1 and on the other hand the fsck command shows that  live Replica: 1 and  excess Replica: 0.
 
Why does the problem is occur?

Does anyone know how to deal with this problem?

Thank you in advance!


Cloudera manager shows this messages:
forum 1.pngforum 2.png
0 REPLIES 0