Created on 11-08-201610:48 AM - edited on 02-29-202011:35 AM by lwang
Many people work exclusively from a laptop where storage space is typically limited to 500GB of space or less. Over time, you may find your available storage space has become a regular concern. It's not uncommon to use an external hard drive to augment available storage space.
The current version of Docker for Mac (1.12.x) does not provide a configuration setting which allows users to change the location where the Docker virtual machine image is located. This means the image, which can grow up to 64GB in size by default, is located on your laptop's primary hard drive.
With the HDP 2.5 version of the Hortonworks sandbox available as a native Docker image, you may find a desire to have more room available to Docker. This tutorial will guide you through the process of moving your Docker virtual machine image to a different location, an external drive in this case. This will free up to 64GB of space on your primary laptop hard drive and let you expand the size of the Docker image file later. This tutorial is the first in a two part series.
Now we are going to stop Docker for Mac. Before shutting down Docker, make sure all of your containers have been stopped. Using the menu shown above, click on the Quit Docker menu option. This will stop Docker for Mac. You should notice the Docker for Mac icon is no longer visible.
Now let's confirm the Docker processes we saw before are no longer running:
NOTE: It may take a minute or two before Docker completely shuts down.
Backup Docker virtual machine image
Before we make any changes to the Docker virtual machine image, we should back it up. This will temporarily use more space on your laptop hard drive. Make sure you have enough room to hold two copies of the data. As mentioned before, the Docker image can be up to 64GB by default. Let's check the current size of our image using du -sh. The Docker image file is located at ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/ by default.
du -sh ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/
In my case, my image size is 64GB. You need to be sure you have room for 2 copies of the com.docker.driver.amd64-linux directory. Now we'll make a copy of our image:
cp -r com.docker.driver.amd64-linux com.docker.driver.amd64-linux.backup
This copy serves as our backup of the image.
Copy Docker virtual machine image to external drive
Now we can make a copy of our image on our external hard drive. I have a 1TB SSD mounted at /Volumes/Samsung. I am going to store my Docker virtual machine image in /Volumes/Samsung/Docker/image. You should store the image in a location that makes sense for you.
This process will take a few minutes. It will take longer if you are not using an SSD. Let's confirm the directory now exists on the external hard drive.
ls -la /Volumes/Samsung/Docker/image/
drwxr-xr-x 3 myoung staff 102 Nov 3 17:08 .
drwxr-xr-x 11 myoung staff 374 Nov 3 17:03 ..
drwxr-xr-x@ 11 myoung staff 374 Nov 7 21:53 com.docker.driver.amd64-linux
You can also check the size:
du -sh /Volumes/Samsung/Docker/image/
Create symbolic link for Docker virtual machine image
Now that we have a copy of the Docker image on the external hard drive, we will use a symbolic link from the image directory on the laptop hard drive to image directory on the external hard drive. Before creating the link, we need to remove the current image directory on our laptop hard drive
Now let's create the symbolic link. We will use the ln -s command. The syntax for ln is ln -s <target> <source>. In this case, target is the location on the external drive and source is the location on the internal drive.
ls -la ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux
lrwxr-xr-x 1 myoung staff 59 Nov 3 17:05 /Users/myoung/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux -> /Volumes/Samsung/Docker/image/com.docker.driver.amd64-linux
Restart Docker for Mac
Now we can restart Docker for Mac. This is done by running the application from the Applications folder in the Finder. You should see something similar to this:
Double-click on the Docker application to start it. You should notice the Docker for Mac icon is now back in the main menu bar. You can also check via ps -ef | grep -i com.docker. You should see something similar to this:
You should notice the Docker processes are running again. You can also check the timestamp of files in the Docker image directory on the external hard drive:
ls -la /Volumes/Samsung/Docker/image/com.docker.driver.amd64-linux
drwxr-xr-x@ 12 myoung staff 408 Nov 7 22:42 .
drwxr-xr-x 3 myoung staff 102 Nov 3 17:08 ..
-rw-r--r-- 1 myoung staff 68676222976 Nov 7 22:45 Docker.qcow2
-rw-r--r-- 1 myoung staff 65536 Nov 7 22:42 console-ring
-rw-r--r-- 1 myoung staff 5 Nov 7 22:42 hypervisor.pid
-rw-r--r-- 1 myoung staff 0 Aug 24 16:06 lock
drwxr-xr-x 67 myoung staff 2278 Nov 5 22:00 log
-rw-r--r-- 1 myoung staff 17 Nov 7 22:42 mac.0
-rw-r--r-- 1 myoung staff 36 Aug 24 16:06 nic1.uuid
-rw-r--r-- 1 myoung staff 5 Nov 7 22:42 pid
-rw-r--r-- 1 myoung staff 59619 Nov 7 22:42 syslog
lrwxr-xr-x 1 myoung staff 12 Nov 7 22:42 tty -> /dev/ttys001
You should notice the timestamp of the Docker.qcow2 file has been updated which means Docker is now using this location for its image file.
Start a Docker container
You should attempt to start a Docker container to make sure everything is working fine. You can start the HDP sandbox via docker start sandbox if you've already installed it as listed in the prerequisites. If everything is working fine, you can delete the backup.
Delete Docker backup image
Now that everything is working using the new location, we can remove our backup.
If you successfully followed along with this tutorial, we were able to move our Docker for Mac virtual machine image to an external hard drive. This should free up to 64GB of space on your laptop hard drive. Look for part two in the series to learn how to increase the size of your Docker image.