Services available out of the box in Hortonworks Data Cloud for AWS

[#1]

The HD cloud documentation states only spark/zeppelin services on data science cluster and spark/hive services on EDW-ETL cluster. Which other HDP 2.5.0 services are included with HD cloud for AWS platform? If these are the only services available, then what options do we have to install and manage other services like HBase, Pig, Oozie etc?

[#2 · Accepted solution]

Thwre are no options to install new services with HDC. It servers only prescriptive use cases and it is not meant to be extended with additional services. For long running clusters in AWS with configurable components and options, please consider using Cloudbreak, HDC was built on Cloudbreak technology. Finally, you can also stand up HDP with AWS by launching AMI and do a manual or blueprint install, this option also allowed for all HDP components we offer.

[#3]

Thanks @ Artem Ervits for conforming.

I have few more queries around HDC/Cloudbreak.

HDC

1. What options do we have to make the application on HDC highly available. Does HDC take care of this?
2. The HDCloud documentation mentions only community support. Is there any other support available for HDC.

Cloudbreak

1. I understand that Cloudbreak automates provision, configure and elastically grow HDP clusters across cloud infrastructure providers. Does it also offer auto downscaling of cluster as per the load?
2. Does Cloudbreak on AWS manage availability?
3. Are there any limitation of Cloudbreak over manual standing up HDP on AWS AMI?

[#4]

HDC

1. HDC is backed by S3, that's the primary way to offer HA. There are no options to deploy HA namenode, RS, etc as clusters are meant to be terminated once used.

2. Plans for enterprise support will be announced in Q1.

Cloudbreak

1. Yes, you can set up alerts to trigger upscale or downscale based on utilization.

2. Yes you can deploy HA across all HA-compatible components.

3. I'd say Cloudbreak makes things easier overall but one could argue you lose some control vs. manual install. Also there might be a short learning curve to learn Cloudbreak but everything is relative. On the flip side, if you learn Cloudbreak once, you can now have a choice to deploy identical infrastructure on another cloud provider. It's all about choice. I recommend you reach out to a local hwx representative who can pull the right team of experts to work through your pains and adivse a solution.