Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on 06-22-2016 10:06 AM - edited 08-18-2019 05:57 AM
HDP 2.4 installed using Ambari 2.2.2.0.
This existing thread mentions clearly the under-replication issue that exists with Accumulo.
Accordingly, I executed several times the below command and also restarted the Accumulo service, the under-replicated blocks dropped to 0 momentarily, then again started to increase :
[hdfs@l4327pp root]$ hdfs dfs -setrep -w 3 /
[hdfs@l4327pp root]$ hdfs fsck / Connecting to namenode via http://l4327pp.sss.com:50070/fsck?ugi=hdfs&path=%2F FSCK started by hdfs (auth:SIMPLE) from /138.106.33.139 for path / at Wed Jun 22 11:56:56 CEST 2016 ....... /apps/accumulo/data/tables/!0/table_info/A000025h.rf: Under replicated BP-736310146-138.106.33.139-1466422465726:blk_1073745134_4310. Target Replicas is 5 but found 4 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s). . /apps/accumulo/data/tables/!0/table_info/A000025i.rf: Under replicated BP-736310146-138.106.33.139-1466422465726:blk_1073745135_4311. Target Replicas is 5 but found 4 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s). . /apps/accumulo/data/tables/!0/table_info/F000025g.rf: Under replicated BP-736310146-138.106.33.139-1466422465726:blk_1073745133_4309. Target Replicas is 5 but found 4 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s). . /apps/accumulo/data/tables/+r/root_tablet/A000025k.rf: Under replicated BP-736310146-138.106.33.139-1466422465726:blk_1073745137_4313. Target Replicas is 5 but found 4 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s). .......................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... ............................................................... /user/accumulo/.Trash/Current/apps/accumulo/data/tables/+r/root_tablet/delete+A000025k.rf+F000025j.rf: Under replicated BP-736310146-138.106.33.139-1466422465726:blk_1073745136_4312. Target Replicas is 5 but found 4 live replica(s), 0 decommissioned replica(s) and 0 decommissioning replica(s). ..................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... ........................................................................Status: HEALTHY Total size: 44223528893 B (Total open files size: 426 B) Total dirs: 139 Total files: 1172 Total symlinks: 0 (Files currently being written: 2) Total blocks (validated): 1490 (avg. block size 29680220 B) (Total open file blocks (not validated): 2) Minimally replicated blocks: 1490 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 5 (0.33557048 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 3 Average block replication: 3.0033557 Corrupt blocks: 0 Missing replicas: 5 (0.11160714 %) Number of data-nodes: 4 Number of racks: 1 FSCK ended at Wed Jun 22 11:56:56 CEST 2016 in 31 milliseconds The filesystem under path '/' is HEALTHY
I even tried to add an entry to the custom accumulo-site.xml and restarted the service but the issue persists :
Created 06-22-2016 06:41 PM
As described in the answer to this question: https://community.hortonworks.com/questions/35138/target-replicas-is-5-but-found-3-live-replicas.htm...
Start the accumulo shell on a host where the Accumulo client is installed and run the following:
config -t accumulo.root -s table.file.replication=3 config -t accumulo.metadata -s table.file.replication=3
You'll have to change these properties AND run the hdfs setrep command so that both new and existing files have their settings changed.
Created 06-22-2016 06:41 PM
As described in the answer to this question: https://community.hortonworks.com/questions/35138/target-replicas-is-5-but-found-3-live-replicas.htm...
Start the accumulo shell on a host where the Accumulo client is installed and run the following:
config -t accumulo.root -s table.file.replication=3 config -t accumulo.metadata -s table.file.replication=3
You'll have to change these properties AND run the hdfs setrep command so that both new and existing files have their settings changed.
Created 06-23-2016 09:25 AM
Accumulo(as always)gave a nasty surprise, was unable to log-in the shell using the credentials I provided during the installation, the tracer failed to start. As per this thread, I kept a simple text password and then executed the commands you provided