question Impact of growing a Datanode Volume in Archives of Support Questions (Read Only)

Impact of growing a Datanode Volume

wsalazar — Tue, 01 Mar 2016 00:01:34 GMT

I think I am asking a slightly different question than is here

https://community.hortonworks.com/questions/6796/how-to-increase-datanode-filesystem-size.html

but a solution should help both.

SAN issues aside!

Is there a method to expand the volume under a datanode directory and have HDFS recognize the new allocated space? For instance if we were to mount a virtual file system, say netapp, in Centos and then expand that filesystem: How would one make the change known to HDFS?

Re: Impact of growing a Datanode Volume

nsabharwal — Tue, 01 Mar 2016 00:59:42 GMT

@wsalazar

You can increase the size and on the safe side run rebalance https://wiki.apache.org/hadoop/FAQ

Re: Impact of growing a Datanode Volume

sshimpi — Tue, 01 Mar 2016 03:57:43 GMT

@wsalazar

I agree with Neeraj.

Yes, You can expand the volume under datanode directory and make it easily available in HDFS.

Two basic things you always needs to take care after increasing/extending existing volume is -

1. OS side : Make sure the new volume is reflecting with newer/extended size [ ie. in linux you can use - partprobe/kpart for lvm =resize2fs, for multipath volume =kpartx ]. Once new size is reflected on OS the HDFS automatically picks up the new size for datanodes without restart required.

2. HDFS side: For evenly distributing data across all datanodes you need to run "Rebalancer" from Cluster UI or command line.