Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Sizing for Master/Edges servers

avatar

I'm helping a prospect expansion from current 6 nodes hadoop cluster to plans of more than 1PB and hundred nodes. I gave him some hints:

- master and edges nodes running in virtual environment (as they do not require high I/O and virtual environment can increase availability)

- knox as security perimeter gateway

- dedicated database nodes with high availability

I need help with recommended sizing and notes for items below:

- Master nodes, what is recommended RAM for master? Prospect asked me to consider that virtualized usually runs on machines with 512GB of RAM and usually they don't allocate more than 64GB virtual hosts.

- Edges nodes

- Knox, do we have any sizing for Knox?

- Database servers, do we have any sizing for dedicated database servers(for metadata: Ambari, Hue, Hive Metastore, Oozie, etc)?

Thanks.

Guilherme.

1 ACCEPTED SOLUTION

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
3 REPLIES 3

avatar

Could you please provide some more details about the services that will be deployed and used? Hive? HBase? Spark?

avatar

@Jonas Straub initially only hive, but in the future Hbase, Solr and Spark also. Prospect does not have all the details yet like number of users, amount of data, etc. So far overall guidelines and basic calculations that lead to number of hosts will help a lot. Thanks.

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login