Support Questions
Find answers, ask questions, and share your expertise

HDP2.5 DATANODE NODEMANGER on different type hosts

Rising Star

We have two types hosts.

Type 1: 64 G memory , 11*900G disks

Type 2: 1T memory, 2*500G disks

The interconnect network is 10GE.

So is following deploy architecture reasonable? Is there any potential risk? Any recommendations? Thanks

DATANODE only on Type 1 hosts

RegionServer also only on Type 1 hosts

NODEMANGER only on Type 2 hosts.

others masters are also on Type 2 hosts.

1 ACCEPTED SOLUTION

Accepted Solutions

Mentor
@Huahua Wei

typically you'd deploy worker (nodemanager) and storage (datanode) service on same nodes to take advantage of having compute close to the data.

View solution in original post

3 REPLIES 3

Mentor
@Huahua Wei

typically you'd deploy worker (nodemanager) and storage (datanode) service on same nodes to take advantage of having compute close to the data.

View solution in original post

Rising Star

Hi Artem, Thanks for your quick response.

Currently, we consider to add some disks to Type 2 hosts(but it still less that the Type 1 hosts) and then add datanode on Type 2 hosts . Can this make the compute close to data? And the HDFS on both 2 types hosts with different size local file systems, Can this acceptable?

Mentor

that can work. What I mean compute close to data means both nodemanager and datanode should be on same node.