Created 06-05-2024 09:58 AM
After installing CDP 7.1.9 on AWS ec2 machines and RDS Postgres as backend database, below databases are created, where scm is the Cloudera Manager database:
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-------------------------+-------------------------+----------+-------------+-------------+-----------------------
amon | amon | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
classic-clusters | classic-clusters | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
cluster-proxy | cluster-proxy | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
das | das | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-alerts | db-alerts | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-clusteraccessmanager | db-clusteraccessmanager | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-dex | db-dex | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-dwx | db-dwx | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-env | db-env | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-liftie | db-liftie | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-mlx | db-mlx | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-resourcepoolmanager | db-resourcepoolmanager | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
db-ums | db-ums | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
hue | hue | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
metastore | hive | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
nav | nav | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
navms | navms | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
oozie | oozie | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
ranger | rangeradmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
rdsadmin | rdsadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | rdsadmin=CTc/rdsadmin
rman | rman | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
schemaregistry | schemaregistry | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
scm | rangeradmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
sentry | sentry | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
ssb_admin | ssb_admin | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
ssb_mve | ssb_mve | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
streamsmsgmgr | streamsmsgmgr | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | rdsadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/rdsadmin +
| | | | | rdsadmin=CTc/rdsadmin
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
What worries us is that all tables are created inside this CM database and other databases are empty:
Clodera Manager (scm) database:
scm=> \dt
List of relations
Schema | Name | Type | Owner
--------+--------------------------------+-------+-------------
public | WM_TRIGGER | table | hive
public | WRITE_SET | table | hive
public | audits | table | scm
public | auth_group | table | hue
public | auth_group_permissions | table | hue
public | auth_permission | table | hue
public | auth_roles | table | scm
public | auth_roles_scopes | table | scm
public | auth_user | table | hue
public | auth_user_groups | table | hue
...
public | beeswax_savedquery | table | hue
public | beeswax_session | table | hue
public | bundle_actions | table | oozie
public | bundle_jobs | table | oozie
...
public | upgrade_state | table | scm
public | user_auth_roles | table | scm
public | user_roles | table | scm
public | user_settings | table | scm
public | useradmin_grouppermission | table | hue
public | useradmin_huepermission | table | hue
public | useradmin_ldapgroup | table | hue
public | useradmin_userprofile | table | hue
public | users | table | scm
public | validate_conn | table | oozie
public | wf_actions | table | oozie
public | wf_jobs | table | oozie
public | x_access_type_def | table | rangeradmin
public | x_audit_metrics | table | rangeradmin
public | x_auth_sess | table | rangeradmin
...
(307 rows)
Values in column owner seems suspicious, as we were expecting the 'rangeradmin' owned tables to go to ranger schema, hive owned to go to metastore db and so on.
When checking those databases, there are no tables at all:
scm=> \c metastore
You are now connected to database "metastore" as user "postgres".
metastore=> \dt
Did not find any relations.
metastore=> \c ranger
You are now connected to database "ranger" as user "postgres".
ranger=> \dt
Did not find any relations.
ranger=> \c ranger rangeradmin
ranger=> \dt
Did not find any relations.
All services Ranger, Hue, Hive are running and writing to the scm database tables.
Questions:
Is this expected behavior?
Where can I find the DDL/SQL codes used during installation?
Created 06-27-2024 10:54 PM
@ldylag No, it is not expected behavior. We expect the tables to be created in the 'public' schema under 'metastore' database.
Created 07-09-2024 05:45 AM
@ldylag You may find the DDLs under /opt/cloudera/parcels/CDH/lib/hive/scripts/metastore/upgrade/<db type>
Created 07-09-2024 08:18 AM
@ldylag you can check which database the service uses by checking its configuration in the cloudera manager. For instance, go to hive service » configuration » and search for database. The configuration named "hive_metastore_database_name" will tell you the actual database hive service is using.
I hope this helps!