question Can't balance datas between disks on a datanode in Support Questions

Can't balance datas between disks on a datanode

OscarData — Thu, 07 Sep 2023 10:07:24 GMT

Hello everybody,

One of my disks on a datanode brake down few months ago. I changed it to a new one but my HDFS won't balance datas between disks. So, i have 7 disks with 80-90% of their capacity and 1 disk with only 15-20% of his capacity.

I tried couple things, such as balancing HDFS or delete every small files.

Furthermore, this difference between disks created a huge amount of blocks in my cluster.

(The datanode has 1,043,627 blocks. Warning threshold : 0)

Do you have any ideas to fix this ?

Thanks,

Oscar

Re: Can't balance datas between disks on a datanode

VidyaSargur — Fri, 08 Sep 2023 09:50:37 GMT

@OscarData, Welcome to our community! To help you get the best possible answer, I have tagged our HDFS/Cloudera Manager experts @vaishaakb @SVB who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Re: Can't balance datas between disks on a datanode

vaishaakb — Fri, 08 Sep 2023 10:28:33 GMT

Hello @OscarData Thank you for reaching out to our community!

Help me understand the following:
Q1. Current CM/CDH Version?

I tried couple things, such as balancing HDFS or delete every small files.

Q2. May I know the results of executing the fore-mentioned tasks? Do you have supporting evidence such as screenshots/snips from logs etc.?

I changed it to a new one

Q3. Do you mind sharing the exact steps of how and what did you change? If there is a documentation you have followed, Please share the link to review and help you efficiently.

Cheers!

- V

Re: Can't balance datas between disks on a datanode

SVB — Mon, 11 Sep 2023 01:35:18 GMT

Hello @OscarData. Running HDFS balancer will not help to balance the disks of a data node. Balancer will only help to balance the data among multiple data nodes but to balance the disks in a particular data node, you would need to run the intra disk balancer. Please review this article on how to run it to balance the disks in a data node. Thanks!

Re: Can't balance datas between disks on a datanode

vaishaakb — Thu, 14 Sep 2023 07:32:22 GMT

@OscarData Circling back to see if you have an update for us on this activity.

- V