Disk Balancer is not enabled

Ghareeb — Sun, 28 Aug 2016 12:01:41 GMT

hi all,

we see the below error in the log file for all cluster machines that dont have balancer role

Disk Balancer is not enabled

how to resolve the issue

Re: Disk Balancer is not enabled

Sailesh Patel — Mon, 29 Aug 2016 07:03:33 GMT

Hi Gareeb,

The error messages that are written to the logs is a known issue in CDH 5.8. These messages are actually harmless, though it is filling up the DataNode logs.
The error message is related to a new feature HDFS-1312 "Re-balance disks within a Datanode". This new feature which allows balancing of data on disks within an individual datanode is not implemented in CDH 5.8 as yet.

The only way forward at this point in time is to suppress this error message and ensure it is not filling the datanode logs. The loglevel of the "org.apache.hadoop.hdfs.server.datanode.DiskBalancer" can be changed to FATAL.

There are two ways to do this. The first requires a restart of the cluster. The second does not require restart but the loglevel is not persisted after restart of datanode.

Option 1) modify log4j settings.

a) navigate to CM -> HDFS -> Configuraion ->DataNode -:> Advanced -> DataNode Logging Advanced Configuration Snippet (Safety Valve)
b) add: log4j.logger.org.apache.hadoop.hdfs.server.datanode.DiskBalancer = FATAL
c) restart DataNodes (rolling restart recommended) .

Option 2) Modify setting on command line.

This will not persist on restarts of the DataNode and you need to run the following command for each DataNode (as 'hdfs' user):

a) Check current logging level:

# hadoop daemonlog -getlevel host-10-17-80-33.coe.cloudera.com:50075 org.apache.hadoop.hdfs.server.datanode.DiskBalancer

b) Set logging level for disk balancer to FATAL

# hadoop daemonlog -setlevel host-10-17-80-33.coe.cloudera.com:50075 org.apache.hadoop.hdfs.server.datanode.DiskBalancer FATAL

I believe this information will explain why this error message is displayed.

Thanks.

Sailesh

Re: Disk Balancer is not enabled

Ghareeb — Mon, 29 Aug 2016 08:47:02 GMT

many thanks

the issue resolved

question Re: Disk Balancer is not enabled in Archives of Support Questions (Read Only)

Disk Balancer is not enabled

Re: Disk Balancer is not enabled

Re: Disk Balancer is not enabled