Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Can't balance datas between disks on a datanode

avatar
New Contributor

Hello everybody,

One of my disks on a datanode brake down few months ago. I changed it to a new one but my HDFS won't balance datas between disks. So, i have 7 disks with 80-90% of their capacity and 1 disk with only 15-20% of his capacity.

OscarData_0-1694079171354.png

I tried couple things, such as balancing HDFS or delete every small files.

Furthermore, this difference between disks created a huge amount of blocks in my cluster. 

OscarData_1-1694079222879.png

(The datanode has 1,043,627 blocks. Warning threshold : 0)

Do you have any ideas to fix this ? 

Thanks,

Oscar

1 ACCEPTED SOLUTION

avatar
Cloudera Employee

Hello @OscarData. Running HDFS balancer will not help to balance the disks of a data node. Balancer will only help to balance the data among multiple data nodes but to balance the disks in a particular data node, you would need to run the intra disk balancer. Please review this article on how to run it to balance the disks in a data node. Thanks!

View solution in original post

4 REPLIES 4

avatar
Community Manager

@OscarData, Welcome to our community! To help you get the best possible answer, I have tagged our HDFS/Cloudera Manager experts @vaishaakb @SVB  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Master Collaborator

Hello @OscarData Thank you for reaching out to our community!

Help me understand the following:
Q1. Current CM/CDH Version?

I tried couple things, such as balancing HDFS or delete every small files.


Q2. May I know the results of executing the fore-mentioned tasks? Do you have supporting evidence such as screenshots/snips from logs etc.?

 

I changed it to a new one

Q3.  Do you mind sharing the exact steps of how and what did you change? If there is a documentation you have followed, Please share the link to review and help you efficiently.

Cheers!

- V

avatar
Cloudera Employee

Hello @OscarData. Running HDFS balancer will not help to balance the disks of a data node. Balancer will only help to balance the data among multiple data nodes but to balance the disks in a particular data node, you would need to run the intra disk balancer. Please review this article on how to run it to balance the disks in a data node. Thanks!

avatar
Master Collaborator

@OscarData Circling back to see if you have an update for us on this activity. 

- V