HDCloud for AWS general availability version 1.14.1 is now available, including six new HDP 2.6 and Ambari 2.5 cluster configurations and new cloud controller features.
If you are new to HDCloud, you can get started using this tutorial (updated for 1.14.1). Officail HDCloud for AWS documentation is available here.
HDP 2.6 and Ambari 2.5
The following HDP 2.6 configurations are now available:
For the list of all available HDP 2.5 and HDP 2.6 configurations, refer to Cluster Configurations documentation.
Resource Tagging
When creating a cluster, you can optionally add custom tags that will be displayed on the CloudFormation stack and on EC2 instances, allowing you to keep track of the resources that cloud controller crates on your behalf. For more information, refer to Resource Tagging documentation.
Node Auto Repair
The cloud controller monitors clusters by checking for Ambari Agent heartbeat on all cluster nodes. If the Ambari Agent heartbeat is lost on a node, a failure is reported for that node. Once the failure is reported, it is fixed automatically (if auto repair is enabled), or options are available for you to fix the failure manually (if auto repair is disabled).
You can configure auto repair settings for each cluster when you create it. For more information, refer to Node Auto Repair documentation.
Auto Scaling
Auto Scaling provides the ability to increase or decrease the number of nodes in a cluster according to the auto scaling policies that you define. After you create an auto scaling policy, cloud controller will execute the policy when the conditions that you specified are met.
You can create an auto scaling policy when creating a cluster or when the cluster is already running you can manage the auto scaling settings and policies. For more information, refer to Auto Scaling documentation.
Protected Gateway
HDCloud now configures a protected gateway on the cluster master node. This gateway is designed to provide access to various cluster resources from a single network port.
Shared Druid Metastore (Technical Preview)
When creating an HDP 2.6 cluster based on the BI configuration, you have an option to have a Druid metastore database created with the cluster, or you can use an external Druid metastore that is backed by Amazon RDS. Using an external Amazon RDS database for a Druid metastore allows you to preserve the Druid metastore metadata and reuse it between clusters. For more information, refer to Managing Shared Metastores documentation.
The features are available via cloud controller UI or CLI.
