Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

[HDP]How Cluster design with two server?

avatar
Rising Star

I want to deploy HDP on two servers. Each server have 32G RAM.

So what best design when install HDP on two servers.

What need assign for Master?

What need assign for Slave?

8999-ho.png

And for practical, is better if use 1 server have 64G RAM or use 2 servers with 32G RAM each?

Thanks.

1 ACCEPTED SOLUTION

avatar
Super Guru

@Hoang Le

I am assuming this is just some sandbox type deployment where you are just trying to experiment and learn Hadoop. Feel free to distribute process evenly on both nodes and set replication factor to 2, making both nodes data nodes. This means, one of your node wil serve as both data node as well as namenode. You cannot have any meaningful Hadoop deployment on two nodes. Not even for POC/evaluation purposes.

You might use these two nodes to install five VMs on it and in that case have two masters and three data nodes.

View solution in original post

3 REPLIES 3

avatar
Super Guru

@Hoang Le

I am assuming this is just some sandbox type deployment where you are just trying to experiment and learn Hadoop. Feel free to distribute process evenly on both nodes and set replication factor to 2, making both nodes data nodes. This means, one of your node wil serve as both data node as well as namenode. You cannot have any meaningful Hadoop deployment on two nodes. Not even for POC/evaluation purposes.

You might use these two nodes to install five VMs on it and in that case have two masters and three data nodes.

avatar
Rising Star

@mqureshi

thanks for your answer.

So about the performance two nodes is the same one node if they have the same total hardware( and also the same if I spare to 5 VMs node) ?

Can you suggest me small cluster design for 3 or 5 node. I want use this on production.

I find the typical cluster referent from document

Masters -- HDFS NameNode, YARN ResourceManager, and HBase Master.

Slaves -- HDFS DataNodes, YARN NodeManagers, and HBase RegionServers

avatar

@Hoang Le If you have 5 nodes, you can use 2 nodes as Masternodes and 3 as Workernodes (slave nodes).