Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 01-31-2018 01:39 PM
we have ambari cluster with 3 masters machine and 4 datanode machine
we run the hdfs dfsadmin -report
and we found Missing blocks: 4
how to know the reason for these missing blocks?
second what is the workaround that we need to do regarding that?
hdfs dfsadmin -report Configured Capacity: 8226130288640 (7.48 TB) Present Capacity: 8225526102776 (7.48 TB) DFS Remaining: 8205621209848 (7.46 TB) DFS Used: 19904892928 (18.54 GB) DFS Used%: 0.24% Under replicated blocks: 4 Blocks with corrupt replicas: 0 Missing blocks: 4 Missing blocks (with replication factor 1): 0 ------------------------------------------------- Live datanodes (4): Name: 10.164.252.32:50010 (worker03.sys76.com) Hostname: worker03.sys76.com Decommission Status : Normal Configured Capacity: 1170504683520 (1.06 TB) DFS Used: 5715611648 (5.32 GB) Non DFS Used: 0 (0 B) DFS Remaining: 1164727208338 (1.06 TB) DFS Used%: 0.49% DFS Remaining%: 99.51% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.33:50010 (worker04.sys76.com) Hostname: worker04.sys76.com Decommission Status : Normal Configured Capacity: 2351875026944 (2.14 TB) DFS Used: 4573270016 (4.26 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2347124950656 (2.13 TB) DFS Used%: 0.19% DFS Remaining%: 99.80% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.31:50010 (worker02.sys76.com) Hostname: worker02.sys76.com Decommission Status : Normal Configured Capacity: 2351875026944 (2.14 TB) DFS Used: 5077798912 (4.73 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2346627408110 (2.13 TB) DFS Used%: 0.22% DFS Remaining%: 99.78% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.30:50010 (worker01.sys76.com) Hostname: worker01.sys76.com Decommission Status : Normal Configured Capacity: 2351875551232 (2.14 TB) DFS Used: 4538212352 (4.23 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2347141642744 (2.13 TB) DFS Used%: 0.19% DFS Remaining%: 99.80% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 8 Last contact: Wed Jan 31 13:34:02 UTC 2018
Created 02-01-2018 08:55 PM
@Michael Bronson , Missing data block can be related to data corruption. Use 'hdfs fsck <path> -list-corruptfileblocks -files -locations' to find out which replicas got corrupted.
Secondly, In order to fix issue, you can delete the corrupted blocks using 'hdfs fsck / -delete'
I hope you find below thread useful for handing missing blocks.
Created 02-01-2018 08:55 PM
@Michael Bronson , Missing data block can be related to data corruption. Use 'hdfs fsck <path> -list-corruptfileblocks -files -locations' to find out which replicas got corrupted.
Secondly, In order to fix issue, you can delete the corrupted blocks using 'hdfs fsck / -delete'
I hope you find below thread useful for handing missing blocks.