Split Cloudera Management Services into 2 boxes

[#1]

I have two boxes to install Cloudera Management Services. How do you recommend to split the services into the two boxes:

The services are:

Service Monitor, Activity Monitor, Host monitor, Reports Manager, Event Server, Alert Publisher. 

[#2 · Accepted solution]

Activity Monitor and Reports Manager use a backend db to store their data. The rest use local directories. So these two are prime to be moved to a second host. You could move the others but you would have to manually migrate the data over.

[#3 · Accepted solution]

Activity Monitor and Reports Manager use a backend db to store their data. The rest use local directories. So these two are prime to be moved to a second host. You could move the others but you would have to manually migrate the data over.

[#4]

CM also uses the backend DB. So you could have CM, Amon, Rman,, and the DB on one host and Smon, Hmon, Event Server and Alert publisher on another.

[#5]

I am building the whole cluster from scratch so I don't have to migrate any data. Also I was planning to use an external database. 

How should I split these services base on how much resources they use (cpu, memory) ? 

[#6 · Accepted solution]

Service Monitor and Activity Monitor are the two heavy hitters. They both write to a TSDB on the local FS in a directory that you specify in the configs.

I would split as I mentioned above:
Host1: CM, DB, Amon, Rman
Host2: Smon, Hmon, Event, Alert

Service Monitor - resource usage dependent on the number of services being ran. It will grow as new nodes are added, as the new nodes will be running new services.

Activity Monitor - Collection information on MR.

Host monitor - Resource req. will depend on the number of hosts; will grow as new hosts are added.

Reports Manager - Is all about the reports. This is usually light if only the pre-built reports are in place and generated. I have had to bump this up as the cluster usage and size grew. I haven't looked into it but it is likely because the reports run on a schedule even if they are not used.

Event Server - Events can be generate out of any metrics or log entries. You can add more or more can be generated if they are triggered by changes to the system.

Alert Publisher - Resource req. will depend on the number of alerts. This will grow with new hosts and services. More alerts can be added.