Member since
09-14-2021
146
Posts
0
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
967 | 09-10-2023 06:35 PM | |
1739 | 08-24-2023 12:08 AM |
09-11-2023
11:30 PM
Hello @newtocm. Yes, You can run your jobs of ingesting and processing data when running the HDFS balancer. But it is ideally recommended to run the balancer when the load on the cluster is not at its peak, so the performance of the jobs are not impacted as balancer is a resource consuming process.
... View more
09-10-2023
06:35 PM
Hello @OscarData. Running HDFS balancer will not help to balance the disks of a data node. Balancer will only help to balance the data among multiple data nodes but to balance the disks in a particular data node, you would need to run the intra disk balancer. Please review this article on how to run it to balance the disks in a data node. Thanks!
... View more
08-24-2023
12:08 AM
@Crash The HDFS balancer works based on the DFS used%. By default the threshold is 10%. So if the DFS used % on a particular data node is greater or lesser than 10% of the average DFS used% across all data nodes, then running the balancer will help to balance the nodes, if the DFS used % is not greater or less than 10% of the average DFS used%, then HDFS will consider the data node to be balanced.
... View more