Skip to main content
rdbb
rdbb
Silver ⭐️⭐️
May 21, 2019
Solved

Is it possible to run manager, utility and edge node on one and the same host?

I have a very small physical cluster of 4 hosts where I want to run a CDH with Cloudera Manager. In order to have at least 3 worker hosts I'd like to install the utility and edge node along with the master node on the same host. Is this possible?
Does it make sense to separete master and utility/edge nodes on two distinct hosts and go with only 2 worker hosts?

Is there an alternative setup being restricted to 4 hosts? Thanks for any advice!

 

 

bgooley
bgooleyAnswer
Gold ⭐️
May 22, 2019

@rdbb ,

 

It is possible to even have a one-node cluster (With CM and CDH), but that is certainly not something that you would want to do if you want to leverage the inherent redundancy and load distrobution of CDH.

 

Functionally, you can put roles most anywhere you please; as far as what is best for your desired use cases, though, we can't really comment without more context.

 

If you only have 4 hosts, you still need to consider what sort of memory the roles that are on those hosts will demand (as well as CPU and disk).  Also, do you want to use High Availability for HDFS and YARN (NameNode and Resource Manager)?

 

Given enough RAM, disk, etc. you could put master host and edge node on one node and then use the other 3 as worker nodes.

 

It is more about what you want to get out of this cluster, how important uptime is, and what resources you have.

If you don't care too much and just want to play around, the configuration I described with master/edge on one host and workers on the other 3 is fine.  In fact, that's basically what we do for our nightly builds here at Cloudera.

 

I hope that halps a bit.  Feel free to ask more questions.

View in thread

6 replies

cabguy0815
cabguy0815
Silver ⭐️
May 22, 2019

Hi,

 

do you want to use this configuration for production or only for evaluation and testing?

 

BR, cabguy0815

Translate
rdbb
rdbbAuthor
Silver ⭐️⭐️
May 22, 2019
Hi @cabguy0815,
it is meant for evaluation and testing but the installation/configuration should not be totally unrealistic. Thanks and best regards, rdbb.
Translate
manuroman
manuroman
Gold ⭐️
May 22, 2019

Hi @rdbb, Its depends of your objetive.

 

There are not any restriction, but really you need a edge node?

 

 

Regards, Manu.

Translate
rdbb
rdbbAuthor
Silver ⭐️⭐️
May 22, 2019

Hi @manuroman ,

 

thanks for the hint. Maybe we can omit the services typically running on an edge node.

But reading Cloudera's CDH Cluster Hosts and Role Assignments my understanding is, that at least three different types of hosts are recommended even for a very small installation.

- One master host

- One host combining utitliy and edge functionality

- Three worker hosts

 

Omitting the edge functionality now comes to the question if it is reasonable to install utilities like the Cloudera Manager on the master host? As mentioned in the original post, we are restricted to 4 hosts.

Best regards,

rdbb

Translate
bgooley
Gold ⭐️
bgooleyAnswer
Gold ⭐️
May 22, 2019

@rdbb ,

 

It is possible to even have a one-node cluster (With CM and CDH), but that is certainly not something that you would want to do if you want to leverage the inherent redundancy and load distrobution of CDH.

 

Functionally, you can put roles most anywhere you please; as far as what is best for your desired use cases, though, we can't really comment without more context.

 

If you only have 4 hosts, you still need to consider what sort of memory the roles that are on those hosts will demand (as well as CPU and disk).  Also, do you want to use High Availability for HDFS and YARN (NameNode and Resource Manager)?

 

Given enough RAM, disk, etc. you could put master host and edge node on one node and then use the other 3 as worker nodes.

 

It is more about what you want to get out of this cluster, how important uptime is, and what resources you have.

If you don't care too much and just want to play around, the configuration I described with master/edge on one host and workers on the other 3 is fine.  In fact, that's basically what we do for our nightly builds here at Cloudera.

 

I hope that halps a bit.  Feel free to ask more questions.

Translate
Terms & ConditionsAccessibility statement