Reply
Explorer
Posts: 6
Registered: ‎05-29-2016
Accepted Solution

Backing up Cloudera Manager database

Hi all,

 

I am trying to backup the Cloudera Manager Database but I can't find how to actually do it. 

I found this:

 

"Cloudera Manager - Contains all the information about services you have configured and their role assignments, all configuration history, commands, users, and running processes. This relatively small database (<100 MB) is the most important to back up."

 

Here is the link: http://www.cloudera.com/documentation/enterprise/5-5-x/topics/cm_ag_backup_dbs.html 

 

Any help is much appreciated. 

Thank you,

D. 

 

Cloudera Employee
Posts: 293
Registered: ‎07-08-2013

Re: Backing up Cloudera Manager database

[ Edited ]

The backup command is RDBMS vendor specific and it depends on your database server (PostgresSQL, MySQL, or Oracle) 

Any idea what is your database type? If you're unsure use the below command to confirm.

# grep -oE 'db.(type|host)=.*' /etc/cloudera-scm-server/db.properties

 

If it's embedded/Postgres, use the guide 'Backing Up PostgreSQL Databases' [1]

 

 

[1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ag_backup_dbs.html#cmig_topic_5_6_...

Explorer
Posts: 6
Registered: ‎05-29-2016

Re: Backing up Cloudera Manager database

[ Edited ]

Hi Michalis,

 

 

It worked, thank you! :) 

 

Explorer
Posts: 14
Registered: ‎08-16-2016

Re: Backing up Cloudera Manager database

[ Edited ]

The document says to run the below as root user from the host running the CM server

 

# pg_dump -h hostname -p 7432 -U scm > /tmp/scm_server_db_backup.$(date +%Y%m%d)

 I have a query here. There is not database specified to be dumped (-d switch). It only specifies the user as scm (-U). How does it dump the Cloudera Manager database?

 

We use the embedded database in our cluster. All databases like Hive metastore, Oozie etc are configured to the same host where Cloudera Manager server runs and embedded postgres database. If I have to backup the other databases, is it enough to provide the user (-U) switch only? Does it dump the database owned by that user only by default?

 

Also would it create any perfomance issues if I dump the Cloudera Manager , Hive and Oozie databases in a production environment?

 

Cloudera Employee
Posts: 293
Registered: ‎07-08-2013

Re: Backing up Cloudera Manager database

[ Edited ]

Hi AnandMS,

 

From what I can tell the pg_dump command expects pg_dump [connection-option...] [option...] [dbname] see

https://www.postgresql.org/docs/9.1/static/app-pgdump.html

 

grep -oE 'db.(user|name)=.*' /etc/cloudera-scm-server/db.properties in my environment give me the followig output.

 

db.name=scm
db.user=cloudera-scm

 

threfore db.user I set in -U cloudera-scm followed by scm.

 

effectively your command should look like

 

# pg_dump -h hostname -p 7432 -U cloudera-scm scm > /tmp/scm_server_db_backup.$(date +%Y%m%d)

 

Explorer
Posts: 14
Registered: ‎08-16-2016

Re: Backing up Cloudera Manager database

Thank you Michalis. It makes sense. I was confused because the db name was not being specified in the Cloudera documentation. 

 

Also do we have to take any precautions before dumping the database (CM , Hive metastore and Oozie dbs)? Does it affect anything if I do it in a cluster when jobs are running?

Explorer
Posts: 14
Registered: ‎08-16-2016

Re: Backing up Cloudera Manager database

Hi Michalis,

 

It worked just fine for me. The dumping process took only a few seconds and there was no impact.

 

Thank you!

Highlighted
New Contributor
Posts: 2
Registered: ‎01-25-2017

Re: Backing up Cloudera Manager database

Hi Michalis,

This solution is not working for me.

I get error like:
pg_dump: symbol lookup error: pg_dump: undefined symbol: PQconnectdbParams

Upon running command:
pg_dump -h hostname -p 7432 -U scm scm > /tmp/scm_server_db_backup.$(date +%Y%m%d)

Could you pls help ?