disk used percentage is not symmetric on datanodes

Zhuo.Chen — Fri, 30 Jan 2015 01:15:27 GMT

hi,

we have a hdfs cluster that have 7 datanodes at first, every datanode have 2TB disk.

lately we add datanode to 20, those new datanodes have 500GB disk. and then the hdfs

cluster had already been balanced.

now the problem is disk used percentage is always not symmetric when the cluster works a period of time,

those 2TB datanodes disk used percentage are about 50%, but the 500GB datanodes disk used percentage

are about 90% , which made disk used percentage warning occurs. we have to balance the cluster manually.

how do I configure the cluster so as to it keep balance automatic?

any help would be appreciated.

Re: disk used percentage is not symmetric on datanodes

denloe — Sun, 01 Feb 2015 16:46:33 GMT

At the present time there is no functionality in HDFS to do a per-disk balancing on a DataNode.

A long-standing Jira has been opened to add this functionality in a future release: