question Re: Options for using HDP2.5 services on AWS in Archives of Support Questions (Read Only)

Options for using HDP2.5 services on AWS

joby_johny — Fri, 16 Sep 2022 10:59:42 GMT

Hi Guys,

I am trying to configure an credit card application which used HDP2.5 and the below applications(not complete list). Right now it is running with hdp2.4 on virtual machine hosted on Azure and need to switch to hdp2.5. Current AWS - Hortonworks Data Controller is limited to a few applications based on roles. Can somebody suggest me how to proceed for hosting this HDP2.5 application in AWS?

Kafka

Storm

Slider

Hive

HBASE

Phenonix

Zookeeper

Regards,

Joby Johny

Re: Options for using HDP2.5 services on AWS

mqureshi — Wed, 01 Feb 2017 01:01:08 GMT

@Joby Johny

You can use Cloudbreak to setup HDP cluster on AWS if HDC does not have everything you need.

http://sequenceiq.com/cloudbreak-docs/release-1.6.1/aws/

Re: Options for using HDP2.5 services on AWS

aervits — Thu, 09 Feb 2017 18:29:05 GMT

You have two choices on Amazon if you plan to run the services listed by you. Hortonworks Data Cloud for AWS is not one of them. It only applies to prescriptive use cases that include ETL, data discovery, data science and soon BI with Hive, Spark, Zeppelin and Druid, respectively. HDC also heavily relies on S3 for long term storage as opposed to hdfs. We did not include many options for the sake of simplicity. For your use case, you can look at Cloudbreak as my colleague noted as it has the similar experience to HDC with the benefit of infinite configurability as well as option to pass your own blueprints which can include the services you listed, including Storm and Kafka. CB relies on hdfs and is used for long running clusters. The other option you have is to spin up AMIs yourself and stand up HDP as you would an on premise cluster. Again both options give you full control and experience as on premise HDP but CB also offers you chance to use spot pricing, burst capacity to spin up or down nodes based on need, automation with CB shell.

Re: Options for using HDP2.5 services on AWS

joby_johny — Thu, 09 Feb 2017 23:49:04 GMT

Thanks Artem. I tried downloading the VMware image , uploaded to s3 ,created the ami and then spin the cluster from ami.