Reply
Posts: 642
Topics: 3
Kudos: 118
Solutions: 67
Registered: ‎08-16-2016

Re: issue with cloudera management services after configuring TLS

Contributor
Posts: 40
Registered: ‎06-12-2017

Re: issue with cloudera management services after configuring TLS

IT WORKS 

i have found my error. The folder were i create the truststore (copy of /usr/lib/jvm/java-7-oracle-cloudera/jre/lib/security/cacerts) must be /var/lib/cloudera-scm-server/. i have done all the selfsigned certificate in this folder (/var/lib/cloudera-scm-server/) and it work.

Contributor
Posts: 40
Registered: ‎06-12-2017

Re: issue with cloudera management services after configuring TLS

now i have acces to Manager

but there is a new message 

WARN 515969315@agentServer-0:org.mortbay.log: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

i have 2 questions: 

normally in which folder trustore must be?

how many trustore must i have?

Contributor
Posts: 40
Registered: ‎06-12-2017

Re: issue with cloudera management services after configuring TLS

[ Edited ]

thanks but But I no longer needed to use rollback

Posts: 642
Topics: 3
Kudos: 118
Solutions: 67
Registered: ‎08-16-2016

Re: issue with cloudera management services after configuring TLS

That warning indicates that something is talking to CM without using SSL. Did you change all of the agent config files to use_tls=1?

As for the truststore questions. First there is a keystore and a truststore. The keystore stores the key and certificate for a service. This is sensitive as it is the source of how a service identifies itself to another. The truststore just hold the signing certificate and is used by clients to trust any certs signed by the certs in it.

The path /usr/lib/jvm/java-7-oracle-cloudera/jre/lib/security/cacerts looks similar to the location that you would store a system-wide truststore. I think that location is right and the name would be jssecacert or something similar. This means that all Java based program will use this by default without needing to tell the app or client of its location.

Now you don't have to use it; you can create and use your own. And you can have as many as you want although each app, service, client can usually only be configured to use one at a time. Plus, since it is only storing the CA cert why not just have them all in one store to cut down the work.

Note: with self-sign certs, the cert itself become the certificate signing or CA cert and must be put in the truststore.
Contributor
Posts: 40
Registered: ‎06-12-2017

Re: issue with cloudera management services after configuring TLS

ok I think I understand

in my case; there are two types of encryption: the first type is for HTTPS and the second is for encryption between agents and server. it means that i must have two keystores? 

if yes how can i send public key from the other client to the truststore?

Contributor
Posts: 40
Registered: ‎06-12-2017

Re: issue with cloudera management services after configuring TLS

2017-07-24 13:22:12,767 WARN 815257673@scm-web-162:org.mortbay.log: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown
org.apache.avro.AvroRemoteException: java.net.ConnectException: Connexion refusée
        at org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:88)
        at com.sun.proxy.$Proxy111.getAvroHealthReports(Unknown Source)
        at com.cloudera.cmf.protocol.firehose.nozzle.TimeoutNozzleIPC.getAvroHealthReports(TimeoutNozzleIPC.java:127)
        at com.cloudera.cmon.NozzleIPCWrapper.getHealthReports(NozzleIPCWrapper.java:599)
        at com.cloudera.server.web.cmf.HealthReportHelper$GetHealthReportCallable.call(HealthReportHelper.java:502)
        at com.cloudera.server.web.cmf.HealthReportHelper.getHealthReport(HealthReportHelper.java:393)
        at com.cloudera.server.web.cmf.HealthCheckController.hostStatusHealthCheckJSON(HealthCheckController.java:427)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:436)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:424)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:574)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:575)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
        at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78)
        at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:131)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at com.jamonapi.http.JAMonServletFilter.doFilter(JAMonServletFilter.java:48)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at com.cloudera.enterprise.JavaMelodyFacade$MonitoringFilter.doFilter(JavaMelodyFacade.java:109)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:125)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.handler.StatisticsHandler.handle(StatisticsHandler.java:53)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connexion refusée
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
        at sun.net.www.http.HttpClient.New(HttpClient.java:308)
        at sun.net.www.http.HttpClient.New(HttpClient.java:326)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1091)
        at org.apache.avro.ipc.HttpTransceiver.writeBuffers(HttpTransceiver.java:71)
        at org.apache.avro.ipc.Transceiver.transceive(Transceiver.java:58)
        at org.apache.avro.ipc.Transceiver.transceive(Transceiver.java:72)
        at org.apache.avro.ipc.Requestor.request(Requestor.java:147)
        at org.apache.avro.ipc.Requestor.request(Requestor.java:101)
        at org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:72)
        ... 77 more

i think this is the cause of this issu.

Contributor
Posts: 40
Registered: ‎06-12-2017

Re: issue with cloudera management services after configuring TLS

all issues are resolve

i forgot to copy the truststore on the other machines of the cluster

thanks for your help

Highlighted
Explorer
Posts: 13
Registered: ‎12-07-2018

Re: issue with cloudera management services after configuring TLS

Below issue can happen if certifcate is expired? I see in some logs that certificates are expired. Please send documentation for certification renewal.

2019-02-13 23:31:58,038 WARN 1168879507@agentServer-54778:org.mortbay.log: javax.net.ssl.SSLException: Received fatal alert: certificate_expired
2019-02-13 23:31:58,703 WARN 1168879507@agentServer-54778:org.mortbay.log: javax.net.ssl.SSLException: Received fatal alert: certificate_expired
2019-02-13 23:32:01,494 INFO 1645307921@scm-web-99151:com.cloudera.server.web.cmf.AuthenticationSuccessEventListener: Authentication success for user: 'admin' from 192.168.10.51
2019-02-13 23:32:03,490 WARN 1168879507@agentServer-54778:org.mortbay.log: javax.net.ssl.SSLException: Received fatal alert: certificate_expired
Announcements