Note: A newer version of this article is available here: https://community.hortonworks.com/articles/194076/using-vagrant-and-virtualbox-to-create-a-local-ins...
Objectives
This tutorial is designed to walk you through the process of using Vagrant and Virtualbox to create a local instance of Cloudbreak. This will allow you to start your Cloudbreak deployer when you want to spin up an HDP cluster on the cloud without incurring costs associated with hosting your Cloudbreak instance on the cloud itself.
Prerequisites
- You should already have installed VirtualBox 5.1.x. Read more here: VirtualBox
- You should already have installed Vagrant 1.9.x. Read more here: Vagrant
- You should already have installed the vagrant-vbguest plugin. This plugin will keep the VirtualBox Guest Additions software current as you upgrade your kernel and/or VirtualBox versions. Read more here: vagrant-vbguest
- You should already have installed the vagrant-hostmanager plugin. This plugin will automatically manage the /etc/hosts file on your local computer and in your virtual machines. Read more here: vagrant-hostmanager
Scope
This tutorial was tested in the following environment:
- macOS Sierra (version 10.12.4)
- VirtualBox 5.1.22
- Vagrant 1.9.4
- vagrant-vbguest plugin 0.14.1
- vagrant-hostnamanger plugin 1.8.6
- Cloudbreak 1.14.0 TP
Steps
Setup Vagrant
Create Vagrant project directory
Before we get started, determine where you want to keep your Vagrant project files. Each Vagrant project should have its own directory. I keep my Vagrant projects in my ~/Development/Vagrant directory. You should also use a helpful name for each Vagrant project directory you create.
$ cd ~/Development/Vagrant $ mkdir centos7-cloudbreak $ cd centos7-cloudbreak
We will be using a CentOS 7.3 Vagrant box, so I include centos7 in the Vagrant project name to differentiate it from a CentOS 6 project. The project is for cloudbreak, so I include that in the name.
Create Vagrantfile
The Vagrantfile tells Vagrant how to configure your virtual machines. You can copy/paste my Vagrantfile below:
# -*- mode: ruby -*-
# vi: set ft=ruby :
# Using yaml to load external configuration files
require 'yaml'
Vagrant.configure("2") do |config|
# Using the hostmanager vagrant plugin to update the host files
config.hostmanager.enabled = true
config.hostmanager.manage_host = true
config.hostmanager.manage_guest = true
config.hostmanager.ignore_private_ip = false
# Loading in the list of commands that should be run when the VM is provisioned.
commands = YAML.load_file('commands.yaml')
commands.each do |command|
config.vm.provision :shell, inline: command
end
# Loading in the VM configuration information
servers = YAML.load_file('servers.yaml')
servers.each do |servers|
config.vm.define servers["name"] do |srv|
srv.vm.box = servers["box"] # Speciy the name of the Vagrant box file to use
srv.vm.hostname = servers["name"] # Set the hostname of the VM
srv.vm.network "private_network", ip: servers["ip"], :adapater=>2 # Add a second adapater with a specified IP
srv.vm.provision :shell, inline: "sed -i'' '/^127.0.0.1\t#{srv.vm.hostname}\t#{srv.vm.hostname}$/d' /etc/hosts" # Remove the extraneous first entry in /etc/hosts
srv.vm.provider :virtualbox do |vb|
vb.name = servers["name"] # Name of the VM in VirtualBox
vb.cpus = servers["cpus"] # How many CPUs to allocate to the VM
vb.memory = servers["ram"] # How much memory to allocate to the VM
end
end
end
end
Create a servers.yaml file
The servers.yaml file contains the configuration information for our VMs. Here is the content from my file:
--- - name: cloudbreak box: bento/centos-7.3 cpus: 2 ram: 4096 ip: 192.168.56.100
NOTE: You may need to modify the IP address to avoid conflicts with your local network.
Create commands.yaml file
The commands.yaml file contains the list of commands that should be run on each VM when they are first provisioned. This allows us to automate configuration tasks that would other wise be tedious and/or repetitive. Here is the content from my file:
- "sudo yum -y update" - "sudo yum -y install net-tools ntp wget lsof unzip tar iptables-services" - "sudo systemctl enable ntpd && sudo systemctl start ntpd" - "sudo systemctl disable firewalld && sudo systemctl stop firewalld" - "sudo iptables --flush INPUT && sudo iptables --flush FORWARD && sudo service iptables save" - "sudo sed -i --follow-symlinks 's/^SELINUX=.*/SELINUX=disabled/g' /etc/sysconfig/selinux"
Start Virtual Machines
Once you have created the 3 files in your Vagrant project directory, you are ready to start your cluster. Creating the cluster for the first time and starting it every time after that uses the same command:
$ vagrant up
You should notice Vagrant automatically updating the packages on the VM.
Once the process is complete you should have 1 servers running. You can verify by looking at the Virtualbox UI where you should see the cloudbreak VM running. You should see something similar to this:
Connect to each virtual machine
You are able to login to the VM via ssh using the vagrant ssh command.
$ vagrant ssh [vagrant@cloudbreak ~]$
Install Cloudbreak
Most of the Cloudbreak installation is covered well in the docs: Cloudbreak Install Docs. However, the first couple of steps in the docs has you install a few packages, change iptables settings, etc. That part of the install is actually handled by the Vagrant provisioning step, so you can skip those steps. You should be able to start at the Docker Service section of the docs.
We need to be root for most of this, so we'll use sudo.
sudo -i
Create Docker Repo
We need to add a repo so we can install Docker.
cat > /etc/yum.repos.d/docker.repo <<"EOF" [dockerrepo] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/7 enabled=1 gpgcheck=1 gpgkey=https://yum.dockerproject.org/gpg EOF
Install Docker Service
Now we need to install Docker and enable the service.
yum install -y docker-engine-1.9.1 docker-engine-selinux-1.9.1 systemctl start docker systemctl enable docker
Install Cloudbreak Deployer
Now we can install Cloudbreak itself.
yum -y install unzip tar curl -Ls s3.amazonaws.com/public-repo-1.hortonworks.com/HDP/cloudbreak/cloudbreak-deployer_1.14.0_$(uname)_x86_64.tgz | sudo tar -xz -C /bin cbd
Once the Cloudbreak Deployer is installed, you can check the version of the install software.
cbd --version
You should see something similar to this:
[root@cloudbreak cloudbreak-deployment]# cbd --version Cloudbreak Deployer: 1.14.0
NOTE: Notice that we are installing version 1.14.0. You may want to consider installing the latest version, which is 1.16.1 as of August 2017.
Create Cloudbreak Profile
You should make a Cloudbreak application directory. This is where the Cloudbreak configuration files and logs will be located.
cd /opt mkdir cloudbreak-deployment cd cloudbreak-deployment
Now you need to setup the Profile file. This file contains environment variables that determines how Cloudbreak runs. Edit Profile using your editor of choice.
I recommend the following settings for your profile:
export UAA_DEFAULT_SECRET='[SECRET]' export UAA_DEFAULT_USER_EMAIL='<myemail>' export UAA_DEFAULT_USER_PW='<mypassword>' export PUBLIC_IP=192.168.56.100 export CLOUDBREAK_SMTP_SENDER_USERNAME='<myemail>' export CLOUDBREAK_SMTP_SENDER_PASSWORD='<mypassword>' export CLOUDBREAK_SMTP_SENDER_HOST='smtp.gmail.com' export CLOUDBREAK_SMTP_SENDER_PORT=25 export CLOUDBREAK_SMTP_SENDER_FROM='<myemail>' export CLOUDBREAK_SMTP_AUTH=true export CLOUDBREAK_SMTP_STARTTLS_ENABLE=true export CLOUDBREAK_SMTP_TYPE=smtp
You should set the UAA_DEFAULT_USER_EMAIL variable to the email address you want to use. This is the account you will use to login to Cloudbreak. You should set the UAA_DEFAULT_USER_PW variable to the password you want to use. This is the password you will use to login to Cloudbreak.
You should set the CLOUDBREAK_SMTP_SENDER_USERNAME variable to the username you use to authenticate to your SMTP server. You should set the CLOUDBREAK_SMTP_SENDER_PASSWORD variable to the password you use to authenticate to your SMTP server.
NOTE: The SMTP variables are how you enable Cloudbreak to send you an email when the cluster operations are done. This is optional and is only required if you want to use the checkbox to get emails when you build a cluster. The example above assumes you are using GMail. You should use the settings appropriate for your SMTP server.
Initialize Cloudbreak Configuration
Now that you have a profile, you can initialize your Cloudbreak configuration files.
cbd generate
You should see something similar to this:
[root@cloudbreak cloudbreak-deployment]# cbd generate * Dependency required, installing sed latest ... * Dependency required, installing jq latest ... * Dependency required, installing docker-compose 1.9.0 ... * Dependency required, installing aws latest ... Unable to find image 'alpine:latest' locally latest: Pulling from library/alpine 03310923a82b: Pulling fs layer 6fc6c6aca926: Pulling fs layer 6fc6c6aca926: Verifying Checksum 6fc6c6aca926: Download complete 03310923a82b: Verifying Checksum 03310923a82b: Download complete 03310923a82b: Pull complete 6fc6c6aca926: Pull complete Digest: sha256:7875e46eb14555e893e7c23a7f90a0d2396f6b56c8c3dcf68f9ed14879b8966c Status: Downloaded newer image for alpine:latest Generating Cloudbreak client certificate and private key in /opt/cloudbreak-deployment/certs. generating docker-compose.yml generating uaa.yml [root@cloudbreak cloudbreak-deployment]#
Start Cloudbreak Deployer
You should be able to start the Cloudbreak Deployer application. This process will first pull down the Docker images used by Cloudbreak.
cbd pull cbd start
You should notice a bunch of images being pulled down:
[root@cloudbreak cloudbreak-deployment]# cbd start generating docker-compose.yml generating uaa.yml Pulling haveged (hortonworks/haveged:1.1.0)... 1.1.0: Pulling from hortonworks/haveged ca26f34d4b27: Pull complete bf22b160fa79: Pull complete d30591ea011f: Pull complete 22615e74c8e4: Pull complete ceb5854e0233: Pull complete Digest: sha256:09f8cf4f89b59fe2b391747181469965ad27cd751dad0efa0ad1c89450455626 Status: Downloaded newer image for hortonworks/haveged:1.1.0 Pulling uluwatu (hortonworks/cloudbreak-web:1.14.0)... 1.14.0: Pulling from hortonworks/cloudbreak-web 16e32a1a6529: Pull complete 8e153fce9343: Pull complete 6af1e6403bfe: Pull complete 075e3418c7e0: Pull complete 9d8191b4be57: Pull complete 38e38dfe826c: Pull complete d5d08e4bc6be: Pull complete 955b472e3e42: Pull complete 02e1b573b380: Pull complete Digest: sha256:06ceb74789aa8a78b9dfe92872c45e045d7638cdc274ed9b0cdf00b74d118fa2 ... Creating cbreak_periscope_1 Creating cbreak_logsink_1 Creating cbreak_identity_1 Creating cbreak_uluwatu_1 Creating cbreak_haveged_1 Creating cbreak_consul_1 Creating cbreak_mail_1 Creating cbreak_pcdb_1 Creating cbreak_uaadb_1 Creating cbreak_cbdb_1 Creating cbreak_sultans_1 Creating cbreak_registrator_1 Creating cbreak_logspout_1 Creating cbreak_cloudbreak_1 Creating cbreak_traefik_1 Uluwatu (Cloudbreak UI) url: https://192.168.56.100 login email: <myemail> password: **** creating config file for hdc cli: /root/.hdc/config
The start command will output the IP address and the username to login which is based on what we setup in the Profile.
Check Cloudbreak Logs
You can always look at the Cloudbreak logs in /opt/cloudbreak-deployment/cbreak.log. You can also use the cbd logs cloudbreak command to view logs in real time. Cloudbreak is ready to use when you see a message similar to Started CloudbreakApplication in 64.156 seconds (JVM running for 72.52).
Login to Cloudbreak
Cloudbreak should now be running. We can login to the UI using the IP address specified in the Profile. In our case that is https://192.168.56.100. Notice Cloudbreak uses https.
You should see a login screen similar to this:
At this point you should be able the Cloudbreak UI screen where you can manage your credentials, blueprints, etc. This tutorial doesn't cover setting up credentials or deploying a cluster. Before you can deploy a cluster you need to setup a platform and credentials. See this link for setting up your credentials:
Stopping Cloudbreak
When you are ready to shutdown Cloudbeak, the process is simple. First you need to stop the Cloudbreak deployer:
$ cbd kill
You should see something similar to this:
[root@cloudbreak cloudbreak-deployment]# cbd kill Stopping cbreak_traefik_1 ... done Stopping cbreak_cloudbreak_1 ... done Stopping cbreak_logspout_1 ... done Stopping cbreak_registrator_1 ... done Stopping cbreak_sultans_1 ... done Stopping cbreak_uaadb_1 ... done Stopping cbreak_cbdb_1 ... done Stopping cbreak_pcdb_1 ... done Stopping cbreak_mail_1 ... done Stopping cbreak_haveged_1 ... done Stopping cbreak_consul_1 ... done Stopping cbreak_uluwatu_1 ... done Stopping cbreak_identity_1 ... done Stopping cbreak_logsink_1 ... done Stopping cbreak_periscope_1 ... done Going to remove cbreak_traefik_1, cbreak_cloudbreak_1, cbreak_logspout_1, cbreak_registrator_1, cbreak_sultans_1, cbreak_uaadb_1, cbreak_cbdb_1, cbreak_pcdb_1, cbreak_mail_1, cbreak_haveged_1, cbreak_consul_1, cbreak_uluwatu_1, cbreak_identity_1, cbreak_logsink_1, cbreak_periscope_1 Removing cbreak_traefik_1 ... done Removing cbreak_cloudbreak_1 ... done Removing cbreak_logspout_1 ... done Removing cbreak_registrator_1 ... done Removing cbreak_sultans_1 ... done Removing cbreak_uaadb_1 ... done Removing cbreak_cbdb_1 ... done Removing cbreak_pcdb_1 ... done Removing cbreak_mail_1 ... done Removing cbreak_haveged_1 ... done Removing cbreak_consul_1 ... done Removing cbreak_uluwatu_1 ... done Removing cbreak_identity_1 ... done Removing cbreak_logsink_1 ... done Removing cbreak_periscope_1 ... done [root@cloudbreak cloudbreak-deployment]#
Now exit the Vagrant box:
[root@cloudbreak cloudbreak-deployment]# exit logout [vagrant@cloudbreak ~]$ exit logout Connection to 127.0.0.1 closed.
Now we can shutdown the Vagrant box
$ vagrant halt ==> cbtest: Attempting graceful shutdown of VM...
Starting Cloudbreak
To startup Cloudbreak, the process is the opposite of stopping it. First you need to start the Vagrant box:
$ vagrant up
Once the Vagrant box is up, you need to ssh in to the box:
$ vagrant ssh
You need to be root:
$ sudo -i
Now start Cloudbreak:
$ cd /opt/cloudbreak-deployment $ cbd start
You should see something similar to this:
[root@cloudbreak cloudbreak-deployment]# cbd start generating docker-compose.yml generating uaa.yml Creating cbreak_consul_1 Creating cbreak_periscope_1 Creating cbreak_sultans_1 Creating cbreak_uluwatu_1 Creating cbreak_identity_1 Creating cbreak_uaadb_1 Creating cbreak_pcdb_1 Creating cbreak_mail_1 Creating cbreak_haveged_1 Creating cbreak_logsink_1 Creating cbreak_cbdb_1 Creating cbreak_logspout_1 Creating cbreak_registrator_1 Creating cbreak_cloudbreak_1 Creating cbreak_traefik_1 Uluwatu (Cloudbreak UI) url: https://192.168.56.100 login email: <myemail> password: **** creating config file for hdc cli: /root/.hdc/config [root@cloudbreak cloudbreak-deployment]#
It takes a minute or two for the Cloudbreak application to fully start up. Now you can login to the Cloudbreak UI.
Review
If you have successfully followed along with this tutorial, you should now have a Vagrant box you can spin up via vagrant up, startup Cloudbreak via cbd start and then create your clusters on the cloud.
