- last edited on
Lets say we have the following cluster with the following details
10 node HDFS cluster, and 4 are of disk size - 10 TB and 6 node of disk size - 1TB
On Hadoop - 2.6, cloudera - 5.8 , we have ability to change the default from round robin to available space in case the disks on data node machine are with different size
Example ( in case of cloudera cluster )
look at dfs.datanode.fsdataset.volume.choosing.policy. By default this is set to round-robin but since you have an asymmetric disk setup you should change it to available space.
Since we have hortonwoks HDP cluster version 2.6.5 ,
we are searching the same abilitySo we search in AMBARI HDFS CONFIG
But we not found the configuration about round-robin / available space in AMBARI HDFS CONFIG
Dose HDP 2.6.5 ambari cluster can give this ability ?
The goal is to balance the data on all disks consider that some disks are small then others
other post with the same discussion - https://community.cloudera.com/t5/Support-Questions/General-guidelines-and-best-practices-for-tuning...