Prerequisites
- You should already have a Cloudbreak v1.14.4 environment running. You can follow this article to create a Cloudbreak instance using Vagrant and Virtualbox: HCC Article
- You should already have credentials created in Cloudbreak for deploying on AWS (or Azure).
Scope
This tutorial was tested in the following environment:
- macOS Sierra (version 10.12.4)
- Cloudbreak 1.14.4
- AWS EC2
NOTE: Cloudbreak 1.14.0 (TP) had a bug which caused HDP 2.6 clusters installs to fail. You should upgrade your Cloudbreak deployer instance to 1.14.4.
Steps
Create application.yml file
UPDATE 05/24/2017: The creation of a custom application.yml file is not required with Cloudbreak 1.14.4. This version of Cloudbreak includes support for HDP 2.5 and HDP 2.6. This step remains for educational purposes for future HDP updates.
You need to create an application.yml file in the etc directory within your Cloudbreak deployment directory. This file will contain the repo information for HDP 2.6. If you followed my tutorial linked above, then your Cloudbreak deployment directory should be /opt/cloudbreak-deployment. If you are using a Cloudbreak instance on AWS or Azure, then your Cloudbreak deployment directory is likely /var/lib/cloudbreak-deployment/.
Edit your <cloudbreak-deployment>/etc/application.yml file using your favorite editor. Copy and paste the following in the file:
cb:
ambari:
repo:
version: 2.5.0.3-7
baseurl: http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.0.3
gpgkey: http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
database:
vendor: embedded
host: localhost
port: 5432
name: postgres
username: ambari
password: bigdata
</p>
<p>
hdp:
entries:
2.5:
version: 2.5.0.1-210
repoid: HDP-2.5
repo:
stack:
repoid: HDP-2.5
redhat6: http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.5.5.0
redhat7: http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0
util:
repoid: HDP-UTILS-1.1.0.21
redhat6: http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6
redhat7: http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7
2.6:
version: 2.6.0.0-598
repoid: HDP-2.6
repo:
stack:
repoid: HDP-2.6
redhat6: http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.0.3
redhat7: http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.0.3
util:
repoid: HDP-UTILS-1.1.0.21
redhat6: http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6
redhat7: http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7
Start Cloudbreak
Once you have created your application.yml file, you can start Cloudbreak.
$ cbd start
NOTE: It may take a couple of minutes before Cloudbreak is fully running.
Create HDP 2.6 Blueprint
To create an HDP 2.6 cluster, we need to update our blueprint to specify HDP 2.6. On the main Cloudbreak UI, click on manage blueprints. You should see something similar to this:
You should see 3 default blueprints. We are going to use the hdp-small-default blueprint as our base. Click on the hdp-small-default blueprint name. You should see something similar to this:
Now click on the blue copy & edit button. You should see something similar to this:
For the Name, you should enter something unqiue and descriptive. I suggest hdp26-small-default. For the Description, you can enter the same information. You should see something similar to this:
Now we need to edit the JSON portion of the blueprint. Scroll down to the bottom of the JSON. You should see something similar to this:
Now edit the blueprint_name value to be hdp26-small-default and edit the stack_version to be 2.6. You should see something similar to this:
Now click on the green create blueprint button. You should see the new blueprint visible in the list of blueprints.
Create HDP 2.6 Small Default Cluster
Now that our blueprint has been created, we can create a cluster and select this blueprint to install HDP 2.6. Select the appropriate credential for your Cloud environment. Click on the create cluster button. You should see something similar to this:
Provide a unique, but descriptive Cluster Name. Ensure you select an appropriate Region. I chose hdp26test as my cluster name and I'm using the US East region:
Now advanced to the next step by clicking on Setup Network and Security. You should see something similar to this:
We don't need to make any changes here, so click on the Choose Blueprint button. You should see something similar to this:
In the Blueprint dropdown, you should see the blueprint we created. Select the hdp26-small-default blueprint. You should see something similar to this:
You need to select which node Ambari will run on. I typically select the master1 node. You should see something similar to this:
Now you can click on the Review and Launch button. You should see something similar to this:
Verify the information presented. If everything looks good, click on the create and start cluster button. Once the cluster build process has started, you should see something similar to this:
Verify HDP Version
Once the cluster has finished building, you can click on the cluster in the Cloudbreak UI. You should see something similar to this:
Click on the Ambari link to load Ambari. Login using the default username and password of admin. Now click on the Admin link in the menu. You should see something similar to this:
Click on the Stack and Versions link. You should see something similar to this:
You should notice that HDP 2.6.0.3 has been deployed.
Review
If you have successfully followed along with this tutorial, you should know how to create/update /etc/application.yml to add specific Ambair and HDP repositories. You should have successfully created an updated blueprint and deployed HDP 2.6 on your cloud of choice.
