Those are the same steps I've taken, except that restarting the -db service did not create a new data directory. Maybe I should re-check the permissions. I've also been working on creating the data directory manually with initdb, etc but somewhere I'm missing a password. Am about to rework pg_hba.conf to let clouderad-scm in without a password. If I can reload the dump (and I used pg_dumpall in  order to get he roles and permissions as well) then I think I can get this thing going. UGH...this has been an amazingly frustrating process. ... View more

OK, further info. The data is still there (it was a long day yesterday, I should have remembered this). By suppressing the 9.5 startup (by modifying its start.conf settings), I was able to manually start the cloudera database as follows (as root)   sudo -u cloudera-scm /usr/lib/postgresql/9.3/bin/postgres -D /var/lib/cloudera-scm-server-db/data -k /var/run/cloudera-scm-server-db &   Get the cloudera-scm password in this file: cat /var/lib/cloudera-scm-server-db/data/generated_password.txt   And then  you can connect, according to the info in /etc/cloudera-scm-server/db.properties which has the ports, etc   /usr/lib/postgresql/9.3/bin/psql -U cloudera-scm -p 7432 -h localhost -d postgres Password for user cloudera-scm: psql (9.3.22) Type "help" for help. postgres=# \du List of roles Role name | Attributes | Member of ---------------------+------------------------------------------------+----------- amon | | {} cloudera-scm | Superuser, Create role, Create DB, Replication | {} hive | | {} hive1 | | {} nav | | {} navms | | {} oozie_oozie_server | | {} oozie_oozie_server1 | | {} oozie_oozie_server2 | | {} oozie_oozie_server3 | | {} rman | | {} scm | | {} postgres=#     I am now going to try to dump the contents of this database, and then upload it to a 9.5 instance in order to convert the stuff. I looked at pg_upgrade(cluster) but these seem to depend on default "main" locations, eg   pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 9.3 main 5432 online postgres /var/lib/postgresql/9.3/main /var/log/postgresql/postgresql-9.3-main.log 9.5 main 5433 down postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log etc, never seem to pick up the -D locations (and they don't seem to have a -D type option)   ... View more

I am stuck in this exact upgrade scenario. I have narrowed down the issue to the embedded postgres server. I used the do-release-upgrade process to move the server from 14.04 to 16.04 and declined to upgrade the postgres server in the process. At the end of it, I have both 9.3 and 9.5 installed and running; however I cannot find the scm or cloudera-scm roles anywhere at all and at this point I am forced to conclude this information has been lost.    Unless the cloudera manager is somehow keeping this stuff somewhere else?  I'm aware of the contents of /etc/cloudera-scm-server/db.properties and of /var/lib/cloudera-scm-server-db/data/generated_password.txt (both of which survived the do-release-upgrade process).   I am planning to reinstall 14.04 and Cloudera and try this again (I am thankfully running this on a test installation as proof of concept) , but FIRST backing up everything in the postgres server BEFORE running the do-release-upgrade on it. And when I say save, I mean everything in the 9.3/main config files as well as a pg_dump of the database and a pg_dumpall of the globals.   I do not know if this will work, but am noting this here for anyone else's possible benefit (I will return here with any further news... there will undoubtedly be more folks landing in this situation of trying to do a release upgrade on a running instance...) ... View more

It seems really strange that the process of adding in a new JournalNode doesn't DO THIS FOR YOU. Basically, if you change or add journalnodes to your setup, nothing will work until/unless you do this behind the scenes stuff. WHY? ... View more

Completely agreed. I had no idea "external db" was meant to be on a server *external to the cluster*. I was doing exactly what Miles described: a single cluster node that is reserved for CM/DB & Namenode management, with all the other nodes as workhorse datanodes (and zookeeper, yarn, etc).  I have reverted to embedded and turned off the warning message for my part, but future users would certainly benefit from that page (especially since it is linked directly from the warning message) to be a lot more detailed.   Thanks, Cindy ... View more

Then how do I change it from embedded to external? I followed the directions given in the warning message *exactly*.   For some update on my situation, as a matter of fact, when I wound up rebooting the server this was on (for other reasons), the discrepancy corrupted the entire cluster installation and I had to reinstall everything. I have left it embedded this time around because honestly with 10 servers in the cluster, this is not something that will overwhelm the database.   But it seems to me if you are going to have this big blinking warning thing and recommendations to switch over to external databases, that set of instructions needs to actually work. What did I miss the first time around that left things in a half and half state? All the configuration on the CM Web  UI had all external databases set up, that tested correctly using the testing buttons, and so on. That apparen tly did not agree with the db.properties file and the ../api/v14/cm/scmDbInfo page info. So what was the missing steps from the procedure outlined in http://tiny.cloudera.com/cm_db_config-5 ? ... View more

From the status/home page of the Cloudera Web UI, I click on Administrations from the tabs along the top. Then choose settings. This gives me a configuration page. Read down the options here, for me the sixth one reads:   Enable Embedded Database Check.    Uncheck it, and then Save Changes (button along the bottom right).     ... View more

Oh thank you. That does help clarify a number of things.    On (2), when I was installing the CM (via the website wizard which steps through each part, including whether or not to use embedded or external databases) I chose external, which is why I find/found the banner so confusing.   As for point (3) I did not run that script in my most recent install (yes, I did this a couple times, starting over once or twice with a fresh OS install each time). But right now, this is what the file says: root@mymachine:~# more /etc/cloudera-scm-server/db.properties # Auto-generated by scm_prepare_database.sh on Wed Feb 8 18:45:16 PST 2017 # # For information describing how to configure the Cloudera Manager Server # to connect to databases, see the "Cloudera Manager Installation Guide." # com.cloudera.cmf.db.type=postgresql com.cloudera.cmf.db.host=localhost com.cloudera.cmf.db.name=scm com.cloudera.cmf.db.user=scm com.cloudera.cmf.db.setupType=EXTERNAL com.cloudera.cmf.db.password=*** root@mymachine:~#   However with regard to your note in (5) I do note that  the .../api/v14/cm/scmDbInfo location  gives me { "scmDbType" : "POSTGRESQL", "embeddedDbUsed" : true }   Hm, now I'm not sure what to think. It seems like something is inconsistent here. Any suggestions?   Thanks again. ... View more