Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Cloud Sizing for AWS, Azure and Google - What are best instance types to use?

Super Guru

What are the best instance types for HDP nodes (master, data, edge)?

I found a number of instance types that may work.

Looking at

TCO

https://awstcocalculator.com/

Rough Pricing

https://calculator.s3.amazonaws.com/index.html

Amazon Types

http://www.ec2instances.info/

Amazon EC2 Types

https://aws.amazon.com/ec2/instance-types/

1 ACCEPTED SOLUTION

Super Guru

@Timothy Spann "best" is a subjective term when it comes to public cloud providers. I would take your typically on perm profile and back port it to the AWS EC2 profiles. From a pricing perspective you can further spread the master/datanodes services to small boxes based on cost difference. For example ZK uses low ram. Then you can find a small 2-4gig box for your ZK quorum. Take the typical on perm requirements and back port into 1:M VM on your cloud provider.

For example

Master Nodes - Multipule of m4.4xlarge, or r3.4xlarge

Data nodes - i2.4xlarge or d2.4xlarge

Storm nodes - c4.4xlarge

Spark - x1.32xlarge

GPU processing - g2.2xlarge

View solution in original post

2 REPLIES 2

Super Guru

@Timothy Spann "best" is a subjective term when it comes to public cloud providers. I would take your typically on perm profile and back port it to the AWS EC2 profiles. From a pricing perspective you can further spread the master/datanodes services to small boxes based on cost difference. For example ZK uses low ram. Then you can find a small 2-4gig box for your ZK quorum. Take the typical on perm requirements and back port into 1:M VM on your cloud provider.

For example

Master Nodes - Multipule of m4.4xlarge, or r3.4xlarge

Data nodes - i2.4xlarge or d2.4xlarge

Storm nodes - c4.4xlarge

Spark - x1.32xlarge

GPU processing - g2.2xlarge

Super Guru

Thanks for the analysis. Does anyone have similiar sizings for Google and Azure:

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.