Cloud Sizing for AWS, Azure and Google - What are best instance types to use?

[#1]

What are the best instance types for HDP nodes (master, data, edge)?

I found a number of instance types that may work.

Looking at

TCO

https://awstcocalculator.com/

Rough Pricing

https://calculator.s3.amazonaws.com/index.html

Amazon Types

http://www.ec2instances.info/

Amazon EC2 Types

https://aws.amazon.com/ec2/instance-types/

[#2 · Accepted solution]

@Timothy Spann "best" is a subjective term when it comes to public cloud providers. I would take your typically on perm profile and back port it to the AWS EC2 profiles. From a pricing perspective you can further spread the master/datanodes services to small boxes based on cost difference. For example ZK uses low ram. Then you can find a small 2-4gig box for your ZK quorum. Take the typical on perm requirements and back port into 1:M VM on your cloud provider.

For example

Master Nodes - Multipule of m4.4xlarge, or r3.4xlarge

Data nodes - i2.4xlarge or d2.4xlarge

Storm nodes - c4.4xlarge

Spark - x1.32xlarge

GPU processing - g2.2xlarge

[#3]

Thanks for the analysis. Does anyone have similiar sizings for Google and Azure: