Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to start cloduera-scm-server after updating certificates

Highlighted

Unable to start cloduera-scm-server after updating certificates

Explorer

We lately updated our certificates on a working CDH cluster, after the Sectigo's legacy AddTrust External CA Root certificate expired on May 30, 2020.

 

When starting the server with 'systemctl start cloudera-scm-server' these errors follow:

 

2020-06-09 16:37:45,585 INFO main:com.cloudera.cmf.security.components.SslHelper: Failed to load SSL trust store and create ssl socket: Keystore was tampered with, or password was incorrect

...

2020-06-09 16:38:13,993 WARN MainThread:org.mortbay.log: failed SslSelectChannelConnector@0.0.0.0:7182: java.io.IOException: Keystore was tampered with, or password was incorrect
2020-06-09 16:38:13,993 WARN MainThread:org.mortbay.log: failed Server@13cba10a: java.io.IOException: Keystore was tampered with, or password was incorrect
2020-06-09 16:38:13,993 ERROR MainThread:com.cloudera.server.cmf.Main: Failed to start Agent listener.
2020-06-09 16:38:13,994 ERROR MainThread:com.cloudera.server.cmf.Main: Server failed.

-----

Now, running 'keytool -list -v -keystore filename.jks' works since the password is known and it is correct.  The cert, intermediate and root were reviewed against what was issued by cert-manager.com and were correct.

 

I would very much like to know what insights others have about this error.

Best Regards - Zane

4 REPLIES 4
Highlighted

Re: Unable to start cloduera-scm-server after updating certificates

Explorer

I was able to resolve the issues with the new certificates:

- I used the new certificates but installed with the original passwords used in setting up the cluster.  The cloudera-scm-server now starts and I am able to log in to the web admin console.  Now there is a new problem: the four services that comprise the Cloudera Management Service will not start.  In the server log there are three internal accounts that will not authenticate:

 

Authentication failure for user: '__cloudera_internal_user__mgmt-SERVICEMONITOR-c85a5f0750364ad8d08892d1cbeb0ba8' from 10.0.0.10

 

Authentication failure for user: '__cloudera_internal_user__mgmt-HOSTMONITOR-c85a5f0750364ad8d08892d1cbeb0ba8' from 10.0.0.10

 

Authentication failure for user: '__cloudera_internal_user__mgmt-SERVICEMONITOR-c85a5f0750364ad8d08892d1cbeb0ba8' from 10.0.0.10

 

This is CDH 5.14.

Highlighted

Re: Unable to start cloduera-scm-server after updating certificates

Expert Contributor

@Zane- 

 

Are these the user accounts set for the DB users? What is the database being used here?

Please confirm if the databases are correctly setup for the management services.

 

Refer https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_mysql.html#concept_dsg_3mq_bl

Highlighted

Re: Unable to start cloduera-scm-server after updating certificates

Explorer

Thank you for the reply.  The database is PostgreSQL, and defined in db.properties.  The database is from the original deployment and never modified or reconfigured.  The cluster was operational until new certificates were installed.

Highlighted

Re: Unable to start cloduera-scm-server after updating certificates

Expert Contributor

@Zane- 

 

It seems that the authentication fails with the database for the three internal accounts.

Please try login using the credentials of these accounts on the postgres database and check if you are able to access the respective management services databases. 

 

If not, please execute DB queries as DB admin to include access for these accounts to the database with the passwords that you use.

 

Refer https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_extrnl_pstgrs.html#cmig_topic_...

Don't have an account?
Coming from Hortonworks? Activate your account here