Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

how to know the reason for missing blocks

avatar

we have ambari cluster with 3 masters machine and 4 datanode machine

we run the hdfs dfsadmin -report

and we found Missing blocks: 4

how to know the reason for these missing blocks?

second what is the workaround that we need to do regarding that?

hdfs dfsadmin -report
Configured Capacity: 8226130288640 (7.48 TB)
Present Capacity: 8225526102776 (7.48 TB)
DFS Remaining: 8205621209848 (7.46 TB)
DFS Used: 19904892928 (18.54 GB)
DFS Used%: 0.24%
Under replicated blocks: 4
Blocks with corrupt replicas: 0
Missing blocks: 4
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (4):
Name: 10.164.252.32:50010 (worker03.sys76.com)
Hostname: worker03.sys76.com
Decommission Status : Normal
Configured Capacity: 1170504683520 (1.06 TB)
DFS Used: 5715611648 (5.32 GB)
Non DFS Used: 0 (0 B)
DFS Remaining: 1164727208338 (1.06 TB)
DFS Used%: 0.49%
DFS Remaining%: 99.51%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 6
Last contact: Wed Jan 31 13:34:02 UTC 2018
Name: 10.164.252.33:50010 (worker04.sys76.com)
Hostname: worker04.sys76.com
Decommission Status : Normal
Configured Capacity: 2351875026944 (2.14 TB)
DFS Used: 4573270016 (4.26 GB)
Non DFS Used: 0 (0 B)
DFS Remaining: 2347124950656 (2.13 TB)
DFS Used%: 0.19%
DFS Remaining%: 99.80%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 6
Last contact: Wed Jan 31 13:34:02 UTC 2018
Name: 10.164.252.31:50010 (worker02.sys76.com)
Hostname: worker02.sys76.com
Decommission Status : Normal
Configured Capacity: 2351875026944 (2.14 TB)
DFS Used: 5077798912 (4.73 GB)
Non DFS Used: 0 (0 B)
DFS Remaining: 2346627408110 (2.13 TB)
DFS Used%: 0.22%
DFS Remaining%: 99.78%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 6
Last contact: Wed Jan 31 13:34:02 UTC 2018
Name: 10.164.252.30:50010 (worker01.sys76.com)
Hostname: worker01.sys76.com
Decommission Status : Normal
Configured Capacity: 2351875551232 (2.14 TB)
DFS Used: 4538212352 (4.23 GB)
Non DFS Used: 0 (0 B)
DFS Remaining: 2347141642744 (2.13 TB)
DFS Used%: 0.19%
DFS Remaining%: 99.80%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 8
Last contact: Wed Jan 31 13:34:02 UTC 2018
Michael-Bronson
1 ACCEPTED SOLUTION

avatar
Guru

@Michael Bronson , Missing data block can be related to data corruption. Use 'hdfs fsck <path> -list-corruptfileblocks -files -locations' to find out which replicas got corrupted.

Secondly, In order to fix issue, you can delete the corrupted blocks using 'hdfs fsck / -delete'

I hope you find below thread useful for handing missing blocks.

https://community.hortonworks.com/questions/17917/best-way-of-handling-corrupt-or-missing-blocks.htm...

View solution in original post

1 REPLY 1

avatar
Guru

@Michael Bronson , Missing data block can be related to data corruption. Use 'hdfs fsck <path> -list-corruptfileblocks -files -locations' to find out which replicas got corrupted.

Secondly, In order to fix issue, you can delete the corrupted blocks using 'hdfs fsck / -delete'

I hope you find below thread useful for handing missing blocks.

https://community.hortonworks.com/questions/17917/best-way-of-handling-corrupt-or-missing-blocks.htm...