Created 01-31-2018 01:39 PM
we have ambari cluster with 3 masters machine and 4 datanode machine
we run the hdfs dfsadmin -report
and we found Missing blocks: 4
how to know the reason for these missing blocks?
second what is the workaround that we need to do regarding that?
hdfs dfsadmin -report Configured Capacity: 8226130288640 (7.48 TB) Present Capacity: 8225526102776 (7.48 TB) DFS Remaining: 8205621209848 (7.46 TB) DFS Used: 19904892928 (18.54 GB) DFS Used%: 0.24% Under replicated blocks: 4 Blocks with corrupt replicas: 0 Missing blocks: 4 Missing blocks (with replication factor 1): 0 ------------------------------------------------- Live datanodes (4): Name: 10.164.252.32:50010 (worker03.sys76.com) Hostname: worker03.sys76.com Decommission Status : Normal Configured Capacity: 1170504683520 (1.06 TB) DFS Used: 5715611648 (5.32 GB) Non DFS Used: 0 (0 B) DFS Remaining: 1164727208338 (1.06 TB) DFS Used%: 0.49% DFS Remaining%: 99.51% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.33:50010 (worker04.sys76.com) Hostname: worker04.sys76.com Decommission Status : Normal Configured Capacity: 2351875026944 (2.14 TB) DFS Used: 4573270016 (4.26 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2347124950656 (2.13 TB) DFS Used%: 0.19% DFS Remaining%: 99.80% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.31:50010 (worker02.sys76.com) Hostname: worker02.sys76.com Decommission Status : Normal Configured Capacity: 2351875026944 (2.14 TB) DFS Used: 5077798912 (4.73 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2346627408110 (2.13 TB) DFS Used%: 0.22% DFS Remaining%: 99.78% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.30:50010 (worker01.sys76.com) Hostname: worker01.sys76.com Decommission Status : Normal Configured Capacity: 2351875551232 (2.14 TB) DFS Used: 4538212352 (4.23 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2347141642744 (2.13 TB) DFS Used%: 0.19% DFS Remaining%: 99.80% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 8 Last contact: Wed Jan 31 13:34:02 UTC 2018
Created 02-01-2018 08:55 PM
@Michael Bronson , Missing data block can be related to data corruption. Use 'hdfs fsck <path> -list-corruptfileblocks -files -locations' to find out which replicas got corrupted.
Secondly, In order to fix issue, you can delete the corrupted blocks using 'hdfs fsck / -delete'
I hope you find below thread useful for handing missing blocks.
Created 02-01-2018 08:55 PM
@Michael Bronson , Missing data block can be related to data corruption. Use 'hdfs fsck <path> -list-corruptfileblocks -files -locations' to find out which replicas got corrupted.
Secondly, In order to fix issue, you can delete the corrupted blocks using 'hdfs fsck / -delete'
I hope you find below thread useful for handing missing blocks.