Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Migrate cloudera management service on other host

Highlighted

Migrate cloudera management service on other host

Explorer

Hello

 Please, help me. How I can move cloudera management service instances (such as EventServer,

Alert Publisher, Host Monitor and Service Monitor) to other host without delete this instances?

Thank You!

10 REPLIES 10

Re: Migrate cloudera management service on other host

Explorer

Hi!

Thanks for posting this, we are now facing a similar issue.

Do you have any fresh insight to share on this?
Thanks and regards,

Filippo

Re: Migrate cloudera management service on other host

Explorer

Hello!

I could not do it without removing cloudera management service. Therefore, the problem is still relevant.

Re: Migrate cloudera management service on other host

Explorer

Hi,

thanks for the prompt reply! Do you mind sharing a brief overview of what you did (something like initial setup; goal; strategy; final setup/result), so that I can start from there and perhaps build from it in case I can think of some idea to try?

It's certainly time consuming and I am sorry to ask you this, but I would appreciate your help immensely :)
Thanks and regards,
Filippo

Re: Migrate cloudera management service on other host

Explorer

Hi, while waiting for your feedback I can share what are my thoughts so far.

 

We have 4 instances in the “management service” group.

Alert Publisher

Event Server

Host Monitor

Service Monitor

 

The configuration of "management service" shows more though: the 4 of them plus

Activity Monitor

Navigator (Audit Server, Metadata Server)

Reports Manager

 

Config for the last 3 items seems not in use (“Requires appropriate Cloudera Enterprise license“ all over) so I assume we don’t have them.

 

I therefore assume we have at most 5 instances to consider:

1 Activity monitor

2 Alert publisher

3 Event server

4 Host monitor

5 Service monitor

 

By looking at https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_ig_installing_configuring_dbs.html... the following relevant database is mentioned

 

Activity Monitor - Contains information about past activities. In large clusters, this database can grow large. Configuring an Activity Monitor database is only necessary if a MapReduce service is deployed.

 

Notice the last bit, even clearer at https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_ig_reqs_space.html 

 

"The Activity Monitor only monitors MapReduce jobs, and does not monitor not YARN applications. If you no longer use MapReduce (MRv1) in your cluster, the Activity Monitor is not required for Cloudera Manager 5 (or higher) or CDH 5 (or higher)"

 

Hence, we might not have to worry about 1 (and in fact I can’t find traces of this thing running, neither in “instances” nor in "ps aux"). In the same link, all 2-5 are mentioned:

2: The Cloudera Management Service role Alert Publisher sources all the content for its work by regularly polling the Event Server for entries that are marked to be sent out using SNMP or SMTP(S). The Alert Publisher is not discussed because it has no noteworthy storage requirements of its own. => no big deal in moving "Alert Publisher" around

3: Default Storage Location: /var/lib/cloudera-scm-eventserver/ on the host where the Event Server role is configured to run.

4-5: Default Storage Location respectively /var/lib/cloudera-host-monitor/ on the host where the Host Monitor role is configured to run. /var/lib/cloudera-service-monitor/ on the host where the Service Monitor role is configured to run. Notice that it also says "After the above migration has taken place, the RDBMS databases previously used by the Service Monitor and Host Monitor are no longer used" which could explain the existence of legacy db config around.

 

Conclusion (&TL;DR)

I THINK in CDH+CM 5.8.2 we can move the cloudera manager service from hostA to hostB by:

i) stopping the cluster and the service (Via CM UI)

ii) installing the service to hostB(without starting it)

iii) copying 3 folders /var/lib/{cloudera-scm-eventserver,cloudera-host-monitor,cloudera-service-monitor} from hostA to hostB

iv) decommission the service manager at hostB

v) start the service manager at hostB

 

Thoughts? :) Filippo

Re: Migrate cloudera management service on other host

Explorer

Hi,

I tried this out on my virtual cluster (2 nodes on VirtualBox running CDH 5.8.2) and it worked!

This is, in details, what I did:

 

  1. copy folders /var/lib/{cloudera-scm-eventserver,cloudera-host-monitor,cloudera-service-monitor} from hostA to hostB (at hostB I used /tmp/ for parking them)
  2. Stop Management Service
  3. Delete Management Service
  4. Add Service (from CM UI) chosing hostB for all instances
  5. Database setup for Activity Monitor: http://www.cloudera.com/documentation/manager/5-1-x/Cloudera-Manager-Installation-Guide/cm5ig_mysql....
    In particular:
    • Create database amon default character set utf8;
    • Grant all on amon.* to ‘amon’@’localhost’ IDENTIFIED BY ‘...’;
    • Grant all on amon.* to ‘amon’@’%’ IDENTIFIED BY ‘...’

6. Move folders from /tmp/ to /var/lib

7. Chown -R cloudera-scm <folder>

8. Chgrp -R cloudera-scm <folder>

9. Start Service

 

Hope it helps,

Filippo

Re: Migrate cloudera management service on other host

Explorer

Hello.

 Yes, it is work. But the condition was to do it without deleting. My goals is cloudera management service working without a break and it had not do it from server console. 

 

Re: Migrate cloudera management service on other host

Explorer

Hi,

"delete" in the UI does not remove the data collected so far (it says it explicitly during the procedure, and I could see this myself). Also, following the backup steps I have in my sketch, I could restart the service over old data, and I could see them in pictures.

Anyway, a decent desktop machine + Virtual Box + virtual nodes are sufficient to try this out live so... good luck :)
Filippo

Re: Migrate cloudera management service on other host

Explorer

Hello.

But monitoring will be stopped on this time.I did not say that the old data is deleted

 

Re: Migrate cloudera management service on other host

Explorer

Hi,

ok then I misunderstood the original assumptions. Sorry about that.

Regards,

Filippo