https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214415#M84500 <P><A rel="user" href="https://community.cloudera.com/users/3418/jsensharma.html" nodeid="3418">@Jay Kumar SenSharma</A></P><P>Thanks for reply, so it only rebalance manually right?</P> Mon, 22 Oct 2018 16:11:20 GMT sen_ke 2018-10-22T16:11:20Z https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214413#M84498 <P>Hi,</P><P>i added 2 new data node to hdfs cluster and in the same rack, will data auto move to new data node?</P><P>my old data node space are 10T per node, new data node space are 20T per node, when old data node space full, new data node only 50% usage right? or will hdfs put more data to large data node? or i need manually excute hdfs balancer when old data node full?</P><P>Thanks.</P> Mon, 22 Oct 2018 16:01:13 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214413#M84498 sen_ke 2018-10-22T16:01:13Z https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214414#M84499 <P><A rel="user" href="https://community.cloudera.com/users/45191/senke.html" nodeid="45191">@Sen Ke</A></P><P>HDFS provides a “balancer” utility to help balance the blocks across DataNodes in the cluster.</P><P>You can run "HDFS Rebalancer" from Ambari UI as described in <A href="https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.0/bk_ambari-operations/content/rebalancing_hdfs.html" target="_blank">https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.0/bk_ambari-operations/content/rebalancing_hdfs.html</A></P><P>To initiate a balancing process, follow these steps:</P><OL> <LI>In Ambari Web, browse to <STRONG>Services &gt; HDFS &gt; Summary</STRONG>.</LI><LI>Click<STRONG> Service Actions</STRONG>, and then click <STRONG>Rebalance HDFS</STRONG>.</LI><LI>Enter the <STRONG>Balance Threshold </STRONG>value as a percentage of disk capacity.</LI><LI>Click<STRONG> Start</STRONG>.</LI></OL><P>You can monitor or cancel a rebalance process by opening the <STRONG>Background Operations</STRONG> window in Ambari.</P><P>.</P><P><STRONG>Reference Article: </STRONG></P><P><A href="https://community.hortonworks.com/articles/87440/hdfs-balancer-balancing-data-between-disks-on-a-da.html" target="_blank">https://community.hortonworks.com/articles/87440/hdfs-balancer-balancing-data-between-disks-on-a-da.html</A></P> Mon, 22 Oct 2018 16:04:18 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214414#M84499 jsensharma 2018-10-22T16:04:18Z https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214415#M84500 <P><A rel="user" href="https://community.cloudera.com/users/3418/jsensharma.html" nodeid="3418">@Jay Kumar SenSharma</A></P><P>Thanks for reply, so it only rebalance manually right?</P> Mon, 22 Oct 2018 16:11:20 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214415#M84500 sen_ke 2018-10-22T16:11:20Z https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214416#M84501 <P><A rel="user" href="https://community.cloudera.com/users/45191/senke.html" nodeid="45191">@Sen Ke</A></P><P>Rebelancing is not something which we want to schedule on hourly or daily basis hence currently there is no out of the box option to schedule it.</P><P>Ambari simply makes it easy by providing any option to run whenever it is needed.</P><P>However if you want to schedule it then you can try using Cron Jobs to do so.</P> Mon, 22 Oct 2018 16:16:26 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214416#M84501 jsensharma 2018-10-22T16:16:26Z https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214417#M84502 <P><A rel="user" href="https://community.cloudera.com/users/3418/jsensharma.html" nodeid="3418">@Jay Kumar SenSharma</A></P><P>i'm tried rebalance hdfs, but failed and show </P><P>0 moved / 0 left / 0 being processed,</P><P>stderr: /var/lib/ambari-agent/data/errors-1216.txt (can not find this log)</P><P>stdout: /var/lib/ambari-agent/data/output-1216.txt (can not find this log)</P><P>looks no work, if need any action before rebalance?</P> Wed, 24 Oct 2018 13:13:04 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/will-data-auto-move-to-new-data-node/m-p/214417#M84502 sen_ke 2018-10-24T13:13:04Z