I have a requirement to periodically restart all cluster nodes at the machine level. Assume I've done an FSCK before starting to confirm that all blocks are fully replicated. Question is, as I restart each node in turn, will the NameNode notice that any block on that node is under-replicated and put those blocks on the replication queue? If this does happen, will it automatically remove those blocks when the data node comes back online and reports it's blocks to the NN? Note, this is a hardware restart, so the Ambari rolling restart doesn't do the job. ... View more

Within a cluster we have no trouble executing commands agains an HA NameNode using the NameServiceID. But it doesn't work when doing discp from one cluster to another because the clusters are unaware of each other's mapping of nodes to NameServiceID. How does one do this? ... View more

We have two use cases--one is the normal slight imbalance that can creep up gradually and the other is when we add new nodes. Ten new nodes can be 100TB+ to move around--it can take a very long time with normal dfs.network.bandwidth.persecond setting. What's a good strategy? Is it reasonable to use chron to reset the value during off hours? What's the best practice? Also, does rebalancing defer to normal processing dynamically? ... View more

The admins want to know why every service has its own account ID, and is there any harm is using the same account for all? The cluster will be tightly secured. What is the best practice? ... View more

Hadoop has long stressed moving the code to the data, both because it's faster to move the code than to move the data, and more importantly because the network is a limited shared resource that can easily be swamped. Erasure coding would seem to require that a large proportion of the data must move across the network because the contents of a single block will reside on multiple nodes. This would presumably apply not just the ToR switch, but the shared network as well, if the ability to tolerate the loss of a rack is preserved. Is this true and how are these principles reconciled? ... View more