API call to Cloudera Manager failed for Cloudera Express

Question

Recently upgraded Director from 6.1 to 6.2.1 and noticed a lot more spam in the logs with the following trace:

[2019-07-22 17:17:27.632 +0000] ERROR [task-thread-10] - - - - - com.cloudera.api.ext.ClouderaManagerClientProxy: API call to Cloudera Manager failed, gathering information javax.ws.rs.NotFoundException: HTTP 404 Not Found at sun.reflect.GeneratedConstructorAccessor246.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.cxf.jaxrs.client.AbstractClient.convertToWebApplicationException(AbstractClient.java:544) at org.apache.cxf.jaxrs.client.ClientProxyImpl.checkResponse(ClientProxyImpl.java:326) at org.apache.cxf.jaxrs.client.ClientProxyImpl.handleResponse(ClientProxyImpl.java:891) at org.apache.cxf.jaxrs.client.ClientProxyImpl.doChainedInvocation(ClientProxyImpl.java:797) at org.apache.cxf.jaxrs.client.ClientProxyImpl.invoke(ClientProxyImpl.java:238) at com.sun.proxy.$Proxy246.readLicense(Unknown Source) at sun.reflect.GeneratedMethodAccessor494.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudera.api.ext.ClouderaManagerClientProxy.invoke(ClouderaManagerClientProxy.java:117) at com.sun.proxy.$Proxy246.readLicense(Unknown Source) at com.cloudera.launchpad.bootstrap.deployment.util.DeploymentRefresher.checkEnterpriseTrial(DeploymentRefresher.java:358) at com.cloudera.launchpad.bootstrap.deployment.util.DeploymentRefresher.refreshCmDetails(DeploymentRefresher.java:298) at com.cloudera.launchpad.bootstrap.deployment.util.DeploymentRefresher.refreshDeploymentEntity(DeploymentRefresher.java:216) at com.cloudera.launchpad.task.RefreshDeployments.doRefresh(RefreshDeployments.java:118) at com.cloudera.launchpad.task.RefreshModelsTask.refreshDeployments(RefreshModelsTask.java:42) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Turning on API Debugging for one of the CMs gives me the following:

2019-07-22 17:16:32,783 DEBUG 994488843@scm-web-4270:com.cloudera.api.ApiExceptionMapper: Exception caught in API invocation. java.util.NoSuchElementException: This installation is currently running Cloudera Express. at com.cloudera.api.dao.impl.LicenseManagerDaoImpl.readLicense(LicenseManagerDaoImpl.java:54) at com.cloudera.api.v1.impl.ClouderaManagerResourceImpl.readLicense(ClouderaManagerResourceImpl.java:52) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180) at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:192) at com.cloudera.api.ApiInvoker.invoke(ApiInvoker.java:116) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:256) at com.cloudera.api.ApiInvoker.invoke(ApiInvoker.java:116) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:256) at com.cloudera.api.ApiInvoker.invoke(ApiInvoker.java:116) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:100) at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:57) at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:93) at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272) at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:239) at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:248) at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:222) at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:153) at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:167) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:286) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:211) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:262) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) at com.cloudera.enterprise.JavaMelodyFacade$MonitoringFilter.doFilter(JavaMelodyFacade.java:109) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at com.cloudera.api.ApiBasicAuthFilter.doFilter(ApiBasicAuthFilter.java:44) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.handler.StatisticsHandler.handle(StatisticsHandler.java:53) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) 2019-07-22 17:16:32,783 DEBUG 994488843@scm-web-4270:com.cloudera.api.ApiInvoker: API Error 404 [/api/v14/cm/license]: ApiErrorMessage{This installation is currently running Cloudera Express.} 2019-07-22 17:16:32,783 DEBUG 994488843@scm-web-4270:com.cloudera.api.ApiInvoker: API Error 404 [/api/v14/cm/license]: ApiErrorMessage{This installation is currently running Cloudera Express.} 2019-07-22 17:16:32,783 DEBUG 994488843@scm-web-4270:com.cloudera.api.ApiInvoker: API Error 404 [/api/v14/cm/license]: ApiErrorMessage{This installation is currently running Cloudera Express.} 2019-07-22 17:16:32,783 DEBUG 994488843@scm-web-4270:com.cloudera.api.LoggingOutInterceptor: API response: ---------- id: 7 Response code: 404

So it looks like this version of Director doesn't respect the Deployment Template containing "License Type: Cloudera Express" when running this task or at least doesn't handle the (expected/unexpected?) 404 nicely!

Just creating this forum post in case anybody else sees the same spam (which with a lot of deployments can be annoying).

Cheers!

Bill Havanki · Accepted Answer

Hi Da,You're correct, you can edit the Altus Director server's logback.xml file to have it not emit log messages for that Java class. It would look like this. <logger name="com.cloudera.api.ext.ClouderaManagerClientProxy" level="INFO" />More details on editing logback.xml are here: https://www.cloudera.com/documentation/director/latest/topics/director_troubleshoot.html#director-intro__d72814e37While those debug lines are noisy, they can be helpful when problems arise. So, in the future, if you are troubleshooting problems with Altus Director and Cloudera Manager, flipping that logger back on might reveal some useful information.

Mike Wilson · Answer

What version of CM are you using?

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.