Member since
09-28-2017
2
Posts
0
Kudos Received
0
Solutions
09-30-2017
01:22 AM
@frank policano May I know what version of HDP are you using? HDFS-6621 and officially released as part of Apache Hadoop 2.6.0. Since this is a bug in the Balancer itself, it is possible to run an updated version of the Balancer without upgrading your cluster. Datanodes will limit the number of threads used for balancing so as to not eat up all the resources of the cluster/datanode. This is what causes the WARN statement you're seeing. By default the number of threads is 5. This was not configurable prior to Apache Hadoop 2.5.0. HDFS-6595added this proeprty dfs.datanode.balance.max.concurrent.moves to allow you to control the number of threads used for balancing. Since this is a datanode side property, this will require an upgrade to your cluster if you want to use this setting. https://stackoverflow.com/questions/25222633/hadoop-balancer-command-warn-messages-threads-quota-is-exceeded Hope this article helps in resolving balancer issue by running from commandline https://community.hortonworks.com/questions/19694/help-with-exception-from-hdfs-balancer.html Hope this helps you.
... View more