Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 09-21-2014 11:24 AM
Hi!
The problem you are getting is a known limitation of Accumulo on small clusters. By default Accumulo attempts to use a replication factor of 5 for the metadata table, ignoring the "table.file.replication" setting. Normally, Cloudera Manager does not set a max replication factor. This causes under-replication warnings until you can correct either the number of nodes or manually adjust the replication setting on that table.
In your cluster, it appears the "dfs.replication.max" setting has been adjusted to match your number of cluster nodes. This is causing Accumulo's attempts to create new files for its internal tables to fail.
Unfortunately, I'm not sure this can be fixed without data loss. However, to recover you should first edit the "dfs.replication.max" setting for HDFS to be >= 5. Then you should adjust the replication on the metadata and root tables to be <= your number of DataNodes. After that it should be safe to lower dfs.replication.max again.
Adjust the replication in the accumulo shell:
$> config -t accumulo.metadata -s table.file.replication=3 $> config -t accumulo.root -s table.file.replication=3