Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDP2.5 DATANODE NODEMANGER on different type hosts

avatar
Expert Contributor

We have two types hosts.

Type 1: 64 G memory , 11*900G disks

Type 2: 1T memory, 2*500G disks

The interconnect network is 10GE.

So is following deploy architecture reasonable? Is there any potential risk? Any recommendations? Thanks

DATANODE only on Type 1 hosts

RegionServer also only on Type 1 hosts

NODEMANGER only on Type 2 hosts.

others masters are also on Type 2 hosts.

1 ACCEPTED SOLUTION

avatar
Master Mentor
@Huahua Wei

typically you'd deploy worker (nodemanager) and storage (datanode) service on same nodes to take advantage of having compute close to the data.

View solution in original post

3 REPLIES 3

avatar
Master Mentor
@Huahua Wei

typically you'd deploy worker (nodemanager) and storage (datanode) service on same nodes to take advantage of having compute close to the data.

avatar
Expert Contributor

Hi Artem, Thanks for your quick response.

Currently, we consider to add some disks to Type 2 hosts(but it still less that the Type 1 hosts) and then add datanode on Type 2 hosts . Can this make the compute close to data? And the HDFS on both 2 types hosts with different size local file systems, Can this acceptable?

avatar
Master Mentor

that can work. What I mean compute close to data means both nodemanager and datanode should be on same node.