09-15-2017 10:42 AM
I'm trying to rebalance hdfs on my data nodes (CDH 5.9.1) after adding a disk. I've followed this blog post: https://blog.cloudera.com/blog/2016/10/how-to-use-the-new-hdfs-intra-datanode-disk-balancer-in-apach... and restarted the cluster. No matter how I format the host (FQDN, hostname only, IP) I always get this response:
[hdfs@ip-10-16-113-100 ~]$ hdfs diskbalancer -plan ip-10-16-113-100 17/09/15 17:34:27 ERROR tools.DiskBalancer: java.lang.IllegalArgumentException
I see no other complaints that I can find, so I'm trying to figure out what I'm doing wrong. Any one gotten this to work that can give me some guidance?
09-18-2017 01:44 PM
After some unknown period of time, the command began working. Current guess is that the system has to do a complete block report before the command becomes availible.
09-24-2018 03:10 PM
09-24-2018 07:48 PM
09-25-2018 10:13 AM
thanks that worked.
I have an 18 data nodes each capacity of 150 TB, current data storage of each DN is around 14TB whereas 1 DN is 4 TB.
I did disc balancing to equally distribute the data but it didn't help. I am thinking it depends on the math % of storage vs allocated.
is there a way i can distribute the data equally now?
09-25-2018 03:58 PM