Created on 01-31-2017 04:53 PM - edited 09-16-2022 03:59 AM
Hi Guys,
I am trying to configure an credit card application which used HDP2.5 and the below applications(not complete list). Right now it is running with hdp2.4 on virtual machine hosted on Azure and need to switch to hdp2.5. Current AWS - Hortonworks Data Controller is limited to a few applications based on roles. Can somebody suggest me how to proceed for hosting this HDP2.5 application in AWS?
Kafka
Storm
Slider
Hive
HBASE
Phenonix
Zookeeper
Regards,
Joby Johny
Created 02-09-2017 03:49 PM
Thanks Artem. I tried downloading the VMware image , uploaded to s3 ,created the ami and then spin the cluster from ami.
Created 01-31-2017 05:01 PM
You can use Cloudbreak to setup HDP cluster on AWS if HDC does not have everything you need.
Created 02-09-2017 10:29 AM
You have two choices on Amazon if you plan to run the services listed by you. Hortonworks Data Cloud for AWS is not one of them. It only applies to prescriptive use cases that include ETL, data discovery, data science and soon BI with Hive, Spark, Zeppelin and Druid, respectively. HDC also heavily relies on S3 for long term storage as opposed to hdfs. We did not include many options for the sake of simplicity. For your use case, you can look at Cloudbreak as my colleague noted as it has the similar experience to HDC with the benefit of infinite configurability as well as option to pass your own blueprints which can include the services you listed, including Storm and Kafka. CB relies on hdfs and is used for long running clusters. The other option you have is to spin up AMIs yourself and stand up HDP as you would an on premise cluster. Again both options give you full control and experience as on premise HDP but CB also offers you chance to use spot pricing, burst capacity to spin up or down nodes based on need, automation with CB shell.
Created 02-09-2017 03:49 PM
Thanks Artem. I tried downloading the VMware image , uploaded to s3 ,created the ami and then spin the cluster from ami.