question Re: [HDP]How Cluster design with two server? in Archives of Support Questions (Read Only)

[HDP]How Cluster design with two server?

hoangletrung — Fri, 16 Sep 2022 10:46:06 GMT

I want to deploy HDP on two servers. Each server have 32G RAM.

So what best design when install HDP on two servers.

What need assign for Master?

What need assign for Slave?

And for practical, is better if use 1 server have 64G RAM or use 2 servers with 32G RAM each?

Thanks.

Re: [HDP]How Cluster design with two server?

mqureshi — Mon, 31 Oct 2016 11:24:07 GMT

@Hoang Le

I am assuming this is just some sandbox type deployment where you are just trying to experiment and learn Hadoop. Feel free to distribute process evenly on both nodes and set replication factor to 2, making both nodes data nodes. This means, one of your node wil serve as both data node as well as namenode. You cannot have any meaningful Hadoop deployment on two nodes. Not even for POC/evaluation purposes.

You might use these two nodes to install five VMs on it and in that case have two masters and three data nodes.

Re: [HDP]How Cluster design with two server?

hoangletrung — Mon, 31 Oct 2016 12:29:45 GMT

@mqureshi

thanks for your answer.

So about the performance two nodes is the same one node if they have the same total hardware( and also the same if I spare to 5 VMs node) ?

Can you suggest me small cluster design for 3 or 5 node. I want use this on production.

I find the typical cluster referent from document

Masters -- HDFS NameNode, YARN ResourceManager, and HBase Master.

Slaves -- HDFS DataNodes, YARN NodeManagers, and HBase RegionServers

Re: [HDP]How Cluster design with two server?

jstraub — Mon, 31 Oct 2016 12:38:20 GMT

@Hoang Le If you have 5 nodes, you can use 2 nodes as Masternodes and 3 as Workernodes (slave nodes).