Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

After disabling TLS/SSL ,cloudera management services are not able to start

avatar
Explorer

Hi,

We were trying to enable TLS on our cluster.We have followed cloudera steps ,but i think missed something.After enabling TLS,we were able to login to port 7183,but all  cloudera management services were disabled.

 

So we reverted back TLS changes from cloudera and removed all keys and certificates.After that given restart to the cloudera manager.

 

After this process,port 7180 started and login page came.But when i am trying to login then it is not redirecting to the GUI.

when i have check the logs. then these exceptions are showing.

 

2019-12-17 13:24:29,320 INFO WebServerImpl:com.cloudera.server.web.cmon.JobDetailGatekeeper: ActivityMonitor configured to allow job details for all jobs.
2019-12-17 13:24:30,850 WARN ActionablesProvider-0:com.cloudera.server.web.cmf.StatusProvider: Failed to submit task for getting status from SERVICE_MONITORING
com.cloudera.cmon.MgmtServiceNotRunningException: SERVICE_MONITORING is not running

 

2019-12-17 13:24:50,238 WARN EventStorePublisherWithRetry-0:com.cloudera.cmf.event.publish.EventStorePublisherWithRetry: Failed to publish event: SimpleEvent{attributes={CATEGORY=[AUDIT_EVENT], SEVERITY=[INFORMATIONAL], SERVICE=[ClouderaManager], SERVICE_TYPE=[ManagerServer], USER=[admin], EVENTCODE=[EV_LOGIN_SUCCESS], MESSAGE_CODES=[LOGIN_SUCCESS]}, content=User admin logged in successfully., timestamp=1576569290230}
2019-12-17 13:24:52,250 INFO 1267052060@scm-web-0:com.cloudera.server.web.cmf.AuthenticationFailureEventListener: Authentication failure for user: '' from 172.26.194.114
2019-12-17 13:25:05,877 INFO 1267052060@scm-web-0:com.cloudera.server.web.cmf.AuthenticationSuccessEventListener: Authentication success for user: 'admin' from 172.26.194.114
2019-12-17 13:25:11,032 INFO CMMetricsForwarder-0:com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder: Failed to send metrics.
java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy122.writeMetrics(Unknown Source)
at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.sendWithAvro(ClouderaManagerMetricsForwarder.java:325)
at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.sendMetrics(ClouderaManagerMetricsForwarder.java:312)
at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.run(ClouderaManagerMetricsForwarder.java:146)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.avro.AvroRemoteException: java.net.ConnectException: Connection refused
at org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:88)
... 11 more
Caused by: java.net.ConnectException: Connection refused

 

Do anyone have any idea on that.Please suggest. 

 

Thanks.

Devendra

 

1 ACCEPTED SOLUTION

avatar
Guru

Hi @pdev ,

 

You may want to check this community thread:

https://community.cloudera.com/t5/Support-Questions/how-to-rollback-cloudera-manager-tls-configurati...

 

It is possible that some of the data was still in CM database which expects TLS login. Could you please run below SQL command to confirm?

select attr, value from CONFIGS where attr in ('web_tls', 'agent_tls');

If you see any returned value is "true" then you need to update the CM database manually. See below steps:

  1. Back up database/table before you makde any further changes
  2. Run the following queries in the CM database:
    Update TLS for web_tls update CONFIGS set value = 'false' where attr = 'web_tls';
    Update TLS for agent_tls update CONFIGS set value = 'false' where attr = 'agent_tls';
  3. In the browser do clear cache including passwords, and cookies before trying to login.

Thanks and hope this helps!

Li Wang, Technical Solution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

View solution in original post

5 REPLIES 5

avatar
Expert Contributor

@pdev have you disabled TLS in the management services as well? After you disable/enable TLS, you will have restart the management services so they communicate with CM with the changed port and protocol.

avatar
Explorer

Hi Rajesh,

We have disabled tls from cloudera admin and removed truststore path and password from cloudera management services.

I hope this what you are asking.if not then please share steps to disable it.

 

Thanks.

Pdev

avatar
Guru

Hi @pdev ,

 

You may want to check this community thread:

https://community.cloudera.com/t5/Support-Questions/how-to-rollback-cloudera-manager-tls-configurati...

 

It is possible that some of the data was still in CM database which expects TLS login. Could you please run below SQL command to confirm?

select attr, value from CONFIGS where attr in ('web_tls', 'agent_tls');

If you see any returned value is "true" then you need to update the CM database manually. See below steps:

  1. Back up database/table before you makde any further changes
  2. Run the following queries in the CM database:
    Update TLS for web_tls update CONFIGS set value = 'false' where attr = 'web_tls';
    Update TLS for agent_tls update CONFIGS set value = 'false' where attr = 'agent_tls';
  3. In the browser do clear cache including passwords, and cookies before trying to login.

Thanks and hope this helps!

Li Wang, Technical Solution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

avatar
Explorer

thanks  a lot Li Wang.It worked .

 

pdev

 

avatar
Guru

Hi @pdev ,

 

Wonderful to hear that! Thanks for marking this thread as resolved!

 

Cheers,

Li

Li Wang, Technical Solution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum