Can't balance datas between disks on a datanode

[#1]

Hello everybody,

One of my disks on a datanode brake down few months ago. I changed it to a new one but my HDFS won't balance datas between disks. So, i have 7 disks with 80-90% of their capacity and 1 disk with only 15-20% of his capacity.

I tried couple things, such as balancing HDFS or delete every small files.

Furthermore, this difference between disks created a huge amount of blocks in my cluster. 

(The datanode has 1,043,627 blocks. Warning threshold : 0)

Do you have any ideas to fix this ? 

Thanks,

Oscar

[#2]

@OscarData, Welcome to our community! To help you get the best possible answer, I have tagged our HDFS/Cloudera Manager experts @vaishaakb @SVB  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

[#3]

Hello @OscarData Thank you for reaching out to our community!

Help me understand the following:
Q1. Current CM/CDH Version?

I tried couple things, such as balancing HDFS or delete every small files.

---

Q2. May I know the results of executing the fore-mentioned tasks? Do you have supporting evidence such as screenshots/snips from logs etc.?

I changed it to a new one

Q3.  Do you mind sharing the exact steps of how and what did you change? If there is a documentation you have followed, Please share the link to review and help you efficiently.

Cheers!

- V

[#4 · Accepted solution]

Hello @OscarData. Running HDFS balancer will not help to balance the disks of a data node. Balancer will only help to balance the data among multiple data nodes but to balance the disks in a particular data node, you would need to run the intra disk balancer. Please review this article on how to run it to balance the disks in a data node. Thanks!

[#5]

@OscarData Circling back to see if you have an update for us on this activity. 

- V