- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Increase size of HDFS.
- Labels:
-
Apache Ambari
-
Apache Hadoop
-
HDFS
Created on ‎07-20-2020 01:05 PM - edited ‎07-20-2020 08:27 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am running Hortonworks Sandbox HDP 2.6.5 on VirtualBox. I have increased the size of my virtual hard disk (.vdi) to 500 GB. However, when I login to Ambari and view the size of my disk, it shows 106 GB only. What should I do to increase the HDFS capacity from 106 GB to 500 GB?
Created on ‎07-23-2020 06:18 AM - edited ‎07-23-2020 06:19 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Since the solution is scattered across many posts, I'm posting a short summary of what I did.
I am running HDP 2.6.5 image on VirtualBox.
- Increased my virtual hard disk through Virtual Media Manager
- In the guest OS,
- Partitioned the unused space
- Formatted the new partition as an ext4 file system
- Mounted the file system
- Update the /etc/fstab (I couldn't do it, as I did not find that file
- In Ambari, under DataNode directory config, added the newly mounted file system as a comma separated value
- Restarted HDFS (my cluster did not have any files, therefore I did not run the below)
Thanks to @Shelton for his guidance.
sudo -u hdfs hdfs balancer
​
Created ‎07-23-2020 02:16 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Great to hear happy hadooping!
Maybe to help other mark the best answer that helped you resolve your problem so other searching for similar solution would use it to resolve similar issues.
Created on ‎07-23-2020 06:18 AM - edited ‎07-23-2020 06:19 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Since the solution is scattered across many posts, I'm posting a short summary of what I did.
I am running HDP 2.6.5 image on VirtualBox.
- Increased my virtual hard disk through Virtual Media Manager
- In the guest OS,
- Partitioned the unused space
- Formatted the new partition as an ext4 file system
- Mounted the file system
- Update the /etc/fstab (I couldn't do it, as I did not find that file
- In Ambari, under DataNode directory config, added the newly mounted file system as a comma separated value
- Restarted HDFS (my cluster did not have any files, therefore I did not run the below)
Thanks to @Shelton for his guidance.
sudo -u hdfs hdfs balancer
​

- « Previous
-
- 1
- 2
- Next »