Hadoop balancer performing very slow in our production cluster.   CDH version 5.5.1 express edition Cluster capacity 850 nodes.   Highest % occupies nodes - 85% Lowest % occupied nodes - 5 %   Recently we have added 40 nodes in our production cluster and initiated the hadoop balancer with following parameters but there is no significant improvement while transfering the data Can you please suggest what are parameters which I need to configure to improve the performance.   Is it possible to transfer TB of data in each iteation.   sudo -u apps hdfs balancer -Ddfs.balancer.movedWinWidth=5400000 \ -Ddfs.balancer.moverThreads=2000 \ -Ddfs.balancer.dispatcherThreads=2000 \ -Ddfs.balance.bandwidthPerSec=200000000 \ -Ddfs.balancer.getBlocks.size=1000000000000 \ -Ddfs.datanode.balance.max.concurrent.moves=24 \ -Ddfs.balancer.getBlocks.min-block-size=536870912 -threshold 15 -include <Datanodes>     Time Stamp               Iteration#  Bytes Already Moved  Bytes Left To Move  Bytes Being Moved Jan 18, 2019 7:44:55 AM       61       568.44 GB           136.22 TB        280 GB Jan 18, 2019 7:52:39 AM       62       576.80 GB           136.22 TB        280 GB Jan 18, 2019 7:58:23 AM       63       583.06 GB           136.21 TB        280 GB Jan 18, 2019 8:04:36 AM       64       590.99 GB           136.21 TB        280 GB Jan 18, 2019 8:10:48 AM       65       602.05 GB           136.24 TB        280 GB         ... View more

In Name node HA scenario ,  if Active Namenode fails and passive Namenode became active, duuring this time how the MR jobs will function. ... View more

Hi Everyone,    1) One of the map attempt task of very critical MR job (which runs every day)  got failed four times -- out of 4 times first attempt got failed  on one host(host120.xxxx.xxxx.xxx) after 2hours 4 min.  but 2nd , 3rd and 4th attempt  were  failed on the same host(host203.xxxx.xxxx.xxx) . The execution time gap among of these 3 attempts like 3 sec.   host203.xxxx.xxxx.xxx having some build issues so 2nd time got failed but ideally 3rd  attempt should be triggered in a different host rather than on same host.   Logs (2nd attempt) WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: java.io.IOException: WritableName can't load class: Caused by: java.io.IOException: WritableName can't load class: at org.apache.hadoop.io.WritableName.getClass(WritableName.java:77) at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:2029)   Please share your  suggestions to fix this issue as this is very critical job.    2 ) Is there any way set the Map reducer configuration  properties from CMD level rather than config files in  Cloudera  or code level or while submitting job  .If so what is the command to set . Kindly share those details ... View more