question Re: Cloud Sizing for AWS, Azure and Google - What are best instance types to use? in Archives of Support Questions (Read Only)

Cloud Sizing for AWS, Azure and Google - What are best instance types to use?

TimothySpann — Fri, 16 Sep 2022 10:31:02 GMT

What are the best instance types for HDP nodes (master, data, edge)?

I found a number of instance types that may work.

Looking at

TCO

https://awstcocalculator.com/

Rough Pricing

https://calculator.s3.amazonaws.com/index.html

Amazon Types

http://www.ec2instances.info/

Amazon EC2 Types

https://aws.amazon.com/ec2/instance-types/

Re: Cloud Sizing for AWS, Azure and Google - What are best instance types to use?

sunile_manjee — Fri, 22 Jul 2016 21:32:08 GMT

@Timothy Spann "best" is a subjective term when it comes to public cloud providers. I would take your typically on perm profile and back port it to the AWS EC2 profiles. From a pricing perspective you can further spread the master/datanodes services to small boxes based on cost difference. For example ZK uses low ram. Then you can find a small 2-4gig box for your ZK quorum. Take the typical on perm requirements and back port into 1:M VM on your cloud provider.

For example

Master Nodes - Multipule of m4.4xlarge, or r3.4xlarge

Data nodes - i2.4xlarge or d2.4xlarge

Storm nodes - c4.4xlarge

Spark - x1.32xlarge

GPU processing - g2.2xlarge

Re: Cloud Sizing for AWS, Azure and Google - What are best instance types to use?

TimothySpann — Sat, 23 Jul 2016 04:17:51 GMT

Thanks for the analysis. Does anyone have similiar sizings for Google and Azure: