Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS + different disk size nodes + data volume choosing policy


HDFS + different disk size nodes + data volume choosing policy

Lets say we have the following cluster with the following details


10 node HDFS cluster, and 4 are of disk size - 10 TB and 6 node of disk size - 1TB


On Hadoop - 2.6, cloudera - 5.8 , we have ability to change the default from round robin to available space in case the disks on data node machine are with different size


Example ( in case of cloudera cluster )

look at dfs.datanode.fsdataset.volume.choosing.policy. By default this is set to round-robin but since you have an asymmetric disk setup you should change it to available space.




Since we have hortonwoks HDP cluster version 2.6.5 ,


we are searching the same ability

So we search in AMBARI HDFS CONFIG

But we not found the configuration about round-robin / available space in AMBARI HDFS CONFIG


Dose HDP 2.6.5 ambari cluster can give this ability ?


The goal is to balance the data on all disks consider that some disks are small then others


other post with the same discussion -


Don't have an account?
Coming from Hortonworks? Activate your account here