Member since
11-20-2018
2
Posts
0
Kudos Received
0
Solutions
11-21-2018
05:33 AM
Peter, we did not received any error logs in name node logs files.
... View more
11-20-2018
05:17 AM
we are running a 4 DataNodes HDFS cluster. below is the disk details of each DataNodes. Node1: Filesystem Size Used Avail Use% Mounted on /dev/sdc1 2.2T 1.5T 571G 73% /hadoop /dev/mapper/rootvg-root_lv 4.0G 3.6G 418M 90% / Node2: Filesystem Size Used Avail Use% Mounted on /dev/sdc 1.1T 254G 791G 25% /hadoop /dev/sdb1 734G 203G 494G 30% /hadoop1 Node3: Filesystem Size Used Avail Use% Mounted on /dev/sdb 1.1T 1015G 30G 98% /hadoop /dev/mapper/rootvg-root_lv 4.0G 3.6G 486M 89% Node4: Filesystem Size Used Avail Use% Mounted on /dev/sdc 1.1T 246G 799G 24% /hadoop /dev/sdb1 734G 200G 497G 29% /hadoop1 we tried HDFS balancer from Ambari-UI as well as from CLI. but we are not able to evenly rebalance the HDFS DataNodes disks. below is the cli command which we used. hdfs balancer -Ddfs.balancer.movedWinWidth=5400000 -Ddfs.balancer.moverThreads=1000 -Ddfs.balancer.dispatcherThreads=200 -Ddfs.datanode.balance.bandwidthPerSec=100000000 -Ddfs.balancer.max-size-to-move=10737418240 -threshold 5 HDFS version is 2.7.1.2.3 any suggestion or advice how we can achieve the rebalancing would be much appreciated.
... View more
Labels:
- Labels:
-
Apache Hadoop