In this tutorial, we will set up the Hortonworks Data Cloud for AWS 1.16 (released in June 2017) using the advanced template, which requires you to configure a VPC and an RDS instance and database prior to launching the cloud controller.
We will go through the following steps:
- Subscribing to HDCloud services
- Launching an RDS instance
- Launching the cloud controller using the advanced template
- Accessing the cloud controller UI
This tutorial assumes no prior experience with AWS. If you encounter errors while performing the steps, refer to the Troubleshooting documentation. Let's get started!
Prerequisites
The prerequisites for this tutorial are:
1. You have an AWS account.
2. You know in which AWS region you want to launch.
3. You have an SSH key pair in the selected region.
If you need help with the steps required to meet these prerequisites, refer to the following post:
How to set up Hortonworks Data Cloud (HDCloud) for AWS
Subscribe to HDCloud Services
In order to use the product, you need to subscribe to two AWS marketplace services. You can access them by searching the https://aws.amazon.com/marketplace/ or by clicking on the following links:
For each of the services, you need to:
1. Open the listing.
2. Click CONTINUE.
3. Click ACCEPT SOFTWARE TERMS.
This will add these two services to Your Software. Now you can launch the cloud controller.
Launch an RDS Instance
As the advanced template requires you to enter information related to an existing RDS instance, you need to create a PostgreSQL RDS instance and a database prior to launching the cloud controller.
Only PostreSQL RDS instance type is supported.
1. Navigate to the RDS Dashboard at https://console.aws.amazon.com/rds.
2. In the top right corner, select the region in which you want to create your DB instance. For simplicity, let’s create the RDS instance in the same region in which you will later launch the cloud controller.
3. In the RDS Dashboard navigation pane, click Instances, and then click Launch DB instance to launch the Launch DB Instance Wizard.
4. In Step 1: Select Engine, select the PostgreSQL Engine and click Select.
5. In Step 2: Production?, select Dev/Test and click Next Step.
6. In Step 3: Specify DB Details, enter:
- For Instance Specifications, you can use values similar to those in the screenshots. Make sure to use DB Engine Version 9.5.4 or later.
- For Settings, come up with an identifier, a username, and a password for your instance.
- Click Next Step.
7. In Step 4: Configure Advanced Settings:
- In the Network & Security section, select the VPC where the RDS instance should be started. I am using the default VPC.
- On the right, in the Connection Information, make sure that the Inbound access on the security group is set to “0.0.0.0/0”. You can change this setting later, but in this stage, the RDS instance must be accessible to the cloud controller that we will create in the next step.
- In the Database Options section, enter a Database Name. This field is not required, so it’s easy to miss it. If you miss it, you will have to create the database manually.
8. Click Launch DB Instance.
9. Click on View Your DB Instances to get redirected to the RDS Dashboard. Keep this page open, as you will need to copy the RDS information and provide it in the CloudFormation template.
While your RDS database is being created, you can get started with the next step, which is launching the cloud controller using the advanced template.
Launch Cloud Controller (Advanced Template)
1. Navigate to the Hortonworks Data Cloud - Controller Service listing page:
2. The only settings that you’ll want to change are:
- The Region, which needs to be the same as the region that you chose in the prerequisites and where you created the RDS.
- The Deployment Options, which needs to be set to ADVANCED.
3. Click Launch with CloudFormation Console and you will be redirected to the Create stack form in the CloudFormation console.
4. On the Select Template page, your template link is already provided, so just click Next.
5. On the Specify Details page, provide the details required:
General Configuration
- Stack name: You can change this if you want to.
- Controller Instance Type: I recommend that you keep the default. If you pick instance type that is not powerful enough, you will run into issues.
- Email Address and Admin Password: You will use these credentials to log in to the cloud controller UI.
Security Configuration
- SSH KeyName: This is your SSH key pair. If you can’t see it in the form, make sure that you are using the same region.
- Remote Access: This should be a range of IP addresses that can reach the cloud controller. If you are just playing around, you can enter “0.0.0.0/0” which will allow access to all; Or you can use this tool http://www.ipaddressguide.com/cidr#range to calculate a valid CIDR range that includes your public IP address.
The parameters under SmartSense Configuration are optional. Enter your SmartSense ID and opt in to SmartSense telemetry if you would like to use flex support.
In order to obtain the remaining parameters, refer to your RDS Dashboard:
Network Configuration
- VPC ID: For simplicity, let’s use the same default VPC as the one used for the RDS instance.
- Subnet ID: Make sure to select a subnet that belongs to the chosen VPC.
RDS Configuration
- RDS Endpoint:You can copy this from the RDS Dashboard.
- RDS Username: You can copy this from the RDS Dashboard (Username).
- RDS Password: This is the password that you chose when creating your RDS instance.
- Database Name: You can copy this from the RDS Dashboard (DB Name). If the DB Name value is blank, you will have to create the database manually.
When done, click Next.
6. On the Options page, under Advanced, you have an option to change the setting for Rollback on Failure. By default, this is set to “Yes”, which means that all of the AWS resources will be deleted if launching the stack fails, and you will avoid being charged for the resources. You can change the setting to “No” if in case of a failure, you want to keep the resources for troubleshooting purposes. Click Next.
7. On the Review page, check I acknowledge that AWS CloudFormation might create IAM resources and then click CREATE.
8. Refresh the CloudFormation console. You will see the status of your stack as CREATE_IN_PROGRESS. If everything goes well, after about 15 minutes, the status will change to CREATE_COMPLETE, at which point you will be able to proceed to the next step.
If you run into any issues, refer to the Troubleshooting documentation.
Get Started with the Cloud Controller UI
1. To access the cloud controller UI, select the stack, click on Outputs, and click on the CloudURL:
2. Even though your browser will tell you that the connection is unsafe, proceed to the UI and log in with the credentials that you provided earlier.
3. After logging in, you will get to the dashboard:
Now you can start creating clusters. Have fun!
Next Steps
To learn more, refer to the HDCloud for AWS product documentation.
Let us know if this was useful and how we can help you in the future.
