Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Options for using HDP2.5 services on AWS

avatar
Contributor

Hi Guys,

I am trying to configure an credit card application which used HDP2.5 and the below applications(not complete list). Right now it is running with hdp2.4 on virtual machine hosted on Azure and need to switch to hdp2.5. Current AWS - Hortonworks Data Controller is limited to a few applications based on roles. Can somebody suggest me how to proceed for hosting this HDP2.5 application in AWS?

Kafka

Storm

Slider

Hive

HBASE

Phenonix

Zookeeper

Regards,

Joby Johny

1 ACCEPTED SOLUTION

avatar
Contributor

Thanks Artem. I tried downloading the VMware image , uploaded to s3 ,created the ami and then spin the cluster from ami.

View solution in original post

3 REPLIES 3

avatar
Super Guru
@Joby Johny

You can use Cloudbreak to setup HDP cluster on AWS if HDC does not have everything you need.

http://sequenceiq.com/cloudbreak-docs/release-1.6.1/aws/

avatar
Master Mentor

You have two choices on Amazon if you plan to run the services listed by you. Hortonworks Data Cloud for AWS is not one of them. It only applies to prescriptive use cases that include ETL, data discovery, data science and soon BI with Hive, Spark, Zeppelin and Druid, respectively. HDC also heavily relies on S3 for long term storage as opposed to hdfs. We did not include many options for the sake of simplicity. For your use case, you can look at Cloudbreak as my colleague noted as it has the similar experience to HDC with the benefit of infinite configurability as well as option to pass your own blueprints which can include the services you listed, including Storm and Kafka. CB relies on hdfs and is used for long running clusters. The other option you have is to spin up AMIs yourself and stand up HDP as you would an on premise cluster. Again both options give you full control and experience as on premise HDP but CB also offers you chance to use spot pricing, burst capacity to spin up or down nodes based on need, automation with CB shell.

avatar
Contributor

Thanks Artem. I tried downloading the VMware image , uploaded to s3 ,created the ami and then spin the cluster from ami.