Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

underreplication blocks

Highlighted

underreplication blocks

Master Collaborator

i am constantly getting these under replicated blocks. .whats the cause and how can I avoid it ?

I clean it regularly and they appear again after a while . My default replication factor is 3 but I see target replicas as 10 ?

.
/user/root/.staging/job_1534884560213_0194/libjars/snappy-java-1.1.1.6.jar:  Under replicated BP-1132026225-10.100.44.17-1534797374843:blk_1073779567_38813. Target Replicas is 10 but found 3 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s).
.
/user/root/.staging/job_1534884560213_0194/libjars/sqoop-1.4.6.2.6.5.0-292.jar:  Under replicated BP-1132026225-10.100.44.17-1534797374843:blk_1073779579_38825. Target Replicas is 10 but found 3 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s).
.
/user/root/.staging/job_1534884560213_0194/libjars/xz-1.5.jar:  Under replicated BP-1132026225-10.100.44.17-1534797374843:blk_1073779589_38835. Target Replicas is 10 but found 3 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s).
....................................
........................................Status: HEALTHY
 Total size:    2382228115 B (Total open files size: 802 B)
 Total dirs:    1325
 Total files:   2340
 Total symlinks:                0 (Files currently being written: 7)
 Total blocks (validated):      2297 (avg. block size 1037104 B) (Total open file blocks (not validated): 6)
 Minimally replicated blocks:   2297 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       705 (30.692207 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              3120 (31.165718 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Tue Sep 25 15:14:51 EDT 2018 in 106 milliseconds

The filesystem under path '/' is HEALTHY
[hdfs@hadoop1 ~]$

2 REPLIES 2

Re: underreplication blocks

Master Collaborator

i cleaned up the under replicated blocks by "hdfs dfs -setrep -w 3" and after 15 mins the underreplicated blocks are back

.
/user/accumulo/.Trash/Current/apps/accumulo/data/tables/+r/root_tablet/delete+A0000odm.rf+F0000odl.rf:  Under replicated BP-1132026225-10.100.44.17-1534797374843:blk_1073787170_46515. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s).
.
/user/accumulo/.Trash/Current/apps/accumulo/data/tables/+r/root_tablet/delete+A0000odr.rf+A0000odm.rf:  Under replicated BP-1132026225-10.100.44.17-1534797374843:blk_1073787171_46516. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s).
.
/user/accumulo/.Trash/Current/apps/accumulo/data/tables/+r/root_tablet/delete+A0000odr.rf+F0000odq.rf:  Under replicated BP-1132026225-10.100.44.17-1534797374843:blk_1073787175_46520. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s).
...........................................

Re: underreplication blocks

Super Collaborator
Don't have an account?
Coming from Hortonworks? Activate your account here