Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Cloudera 7.1.9 with PostgreSQL on RDS creates all tables in CM database instead of other databases

avatar
New Contributor

After installing CDP 7.1.9 on AWS ec2 machines and RDS Postgres as backend database, below databases are created, where scm is the Cloudera Manager database:

                                                List of databases
          Name           |          Owner          | Encoding |   Collate   |    Ctype    |   Access privileges
-------------------------+-------------------------+----------+-------------+-------------+-----------------------
 amon                    | amon                    | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 classic-clusters        | classic-clusters        | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 cluster-proxy           | cluster-proxy           | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 das                     | das                     | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-alerts               | db-alerts               | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-clusteraccessmanager | db-clusteraccessmanager | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-dex                  | db-dex                  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-dwx                  | db-dwx                  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-env                  | db-env                  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-liftie               | db-liftie               | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-mlx                  | db-mlx                  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-resourcepoolmanager  | db-resourcepoolmanager  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 db-ums                  | db-ums                  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 hue                     | hue                     | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 metastore               | hive                    | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 nav                     | nav                     | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 navms                   | navms                   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 oozie                   | oozie                   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres                | postgres                | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 ranger                  | rangeradmin             | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 rdsadmin                | rdsadmin                | UTF8     | en_US.UTF-8 | en_US.UTF-8 | rdsadmin=CTc/rdsadmin
 rman                    | rman                    | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 schemaregistry          | schemaregistry          | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 scm                     | rangeradmin             | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 sentry                  | sentry                  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 ssb_admin               | ssb_admin               | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 ssb_mve                 | ssb_mve                 | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 streamsmsgmgr           | streamsmsgmgr           | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0               | rdsadmin                | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/rdsadmin          +
                         |                         |          |             |             | rdsadmin=CTc/rdsadmin
 template1               | postgres                | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
                         |                         |          |             |             | postgres=CTc/postgres

What worries us is that all tables are created inside this CM database and other databases are empty:

Clodera Manager (scm) database:

scm=> \dt
                       List of relations
 Schema |              Name              | Type  |    Owner
--------+--------------------------------+-------+-------------
 public | WM_TRIGGER                     | table | hive
 public | WRITE_SET                      | table | hive
 public | audits                         | table | scm
 public | auth_group                     | table | hue
 public | auth_group_permissions         | table | hue
 public | auth_permission                | table | hue
 public | auth_roles                     | table | scm
 public | auth_roles_scopes              | table | scm
 public | auth_user                      | table | hue
 public | auth_user_groups               | table | hue
...
 public | beeswax_savedquery             | table | hue
 public | beeswax_session                | table | hue
 public | bundle_actions                 | table | oozie
 public | bundle_jobs                    | table | oozie
...
 public | upgrade_state                  | table | scm
 public | user_auth_roles                | table | scm
 public | user_roles                     | table | scm
 public | user_settings                  | table | scm
 public | useradmin_grouppermission      | table | hue
 public | useradmin_huepermission        | table | hue
 public | useradmin_ldapgroup            | table | hue
 public | useradmin_userprofile          | table | hue
 public | users                          | table | scm
 public | validate_conn                  | table | oozie
 public | wf_actions                     | table | oozie
 public | wf_jobs                        | table | oozie
 public | x_access_type_def              | table | rangeradmin
 public | x_audit_metrics                | table | rangeradmin
 public | x_auth_sess                    | table | rangeradmin
...
(307 rows)

Values in column owner seems suspicious, as we were expecting the 'rangeradmin' owned tables to go to ranger schema, hive owned to go to metastore db and so on.
When checking those databases, there are no tables at all:

scm=> \c metastore
You are now connected to database "metastore" as user "postgres".
metastore=> \dt
Did not find any relations.

metastore=> \c ranger
You are now connected to database "ranger" as user "postgres".
ranger=> \dt
Did not find any relations.

ranger=> \c ranger rangeradmin
ranger=> \dt
Did not find any relations.

All services Ranger, Hue, Hive are running and writing to the scm database tables.

Questions:
Is this expected behavior?
Where can I find the DDL/SQL codes used during installation?

 

3 REPLIES 3

avatar
Master Collaborator

@ldylag No, it is not expected behavior. We expect the tables to be created in the 'public' schema under 'metastore' database.

avatar
Master Collaborator

@ldylag You may find the DDLs under /opt/cloudera/parcels/CDH/lib/hive/scripts/metastore/upgrade/<db type>

avatar
Super Collaborator

@ldylag you can check which database the service uses by checking its configuration in the cloudera manager. For instance, go to hive service » configuration » and search for database. The configuration named "hive_metastore_database_name" will tell you the actual database hive service is using. 

I hope this helps!