Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to increase hdfs storage capacity

how to increase hdfs storage capacity

Expert Contributor

hi i have 5 node each node have 1 To of storage , but i found that each one use just 50 GB

how to increase the capacity storage for each one

[root@Manager ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_manager-lv_root 50G 39G 8.2G 83% / tmpfs 3.8G 80K 3.8G 1% /dev/shm /dev/sda1 477M 41M 411M 9% /boot /dev/mapper/vg_manager-lv_home 860G 7.6G 809G 1% /home

4 REPLIES 4
Highlighted

Re: how to increase hdfs storage capacity

@Mourad Chahri

Your FS layout uses 860 GB for /home directories. I assume that when you installed operating system you used the default file system layout instead of manually defining the layout as required by the HDP installation. You could reallocate that /home space and follow HDP installation instructions. This is a destructive action because you already have a lot of home directories for HDP service accounts and you would have to unmount home and then reallocate the available space for HDP and then remount a small /home. You would have to backup the folders in /home before the unmount and copy them back after smaller /home remount. It can be done, but it is more work than reinstalling OS with proper layout for HDP installation.

Look here to see how the file system should be partitioned:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_cluster-planning-guide/content/ch_partit...

If this response was helpful, please vote and accept it as the best answer.

Highlighted

Re: how to increase hdfs storage capacity

Expert Contributor

yes i use the default file system ,i see that they are other solution , i can add a new directory to hdfs-site into dfs.datanode.data.dir but i dont know how to do it

Highlighted

Re: how to increase hdfs storage capacity

Rising Star
@Mourad Chahri

To add a new directory to HDFS, use the following community question for the same :

https://community.hortonworks.com/questions/49340/how-do-i-change-namenode-and-datanode-dir-for-an-e.... This has a certified approved answer.

You can also check :

https://community.hortonworks.com/articles/2308/how-to-move-or-change-the-hdfs-datanode-directorie.h...

Hope this helps!

Highlighted

Re: how to increase hdfs storage capacity

Expert Contributor

i know how to create hdfs directory , but how to mount a point on this directory and add it to dfs.datanode.data.dir

Don't have an account?
Coming from Hortonworks? Activate your account here