About kshiva

kshiva · ‎08-16-2017

You could use telegraf as metrics collector and sink the data nodes (volume information) metrics (also all master services metrics if need be) to graphite. Then, you can use graphite as a data source in grafana to alert on volume failures. This is an end to end enterprise solution if you want to start monitoring your cluster. Telegraf provided out of the box solution to monitor host level services as well.

kshiva · ‎04-01-2017

Thanks Mugdha. All the steps were followed initially except ambari-server setup-security. After this was run, the UI came up.

kshiva · ‎03-20-2017

Adapter operation failed » 500: Failed to fetch results by the proxy from url: http://hostname:8188/ws/v1/timeline/TEZ_DAG_ID?limit=11&_=1490004805683. Internal Error.. SPNego authentication failed, can not get hadoop.auth cookie Details: { "message": "Failed to fetch results by the proxy from url: http://hostname:8188/ws/v1/timeline/TEZ_DAG_ID?limit=11&_=1490004805683. Internal Error.. SPNego authentication failed, can not get hadoop.auth cookie", "status": 500, "trace": "java.io.IOException: SPNego authentication failed, can not get hadoop.auth cookie\n\tat org.apache.ambari.server.controller.internal.AppCookieManager.getAppCookie(AppCookieManager.java:123)\n\tat org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:228)\n\tat org.apache.ambari.server.view.ViewURLStreamProvider.getHttpURLConnection(ViewURLStreamProvider.java:239)\n\tat org.apache.ambari.server.view.ViewURLStreamProvider.getConnection(ViewURLStreamProvider.java:146)\n\tat org.apache.ambari.server.view.ViewURLStreamProvider.getConnectionAs(ViewURLStreamProvider.java:163)\n\tat org.apache.ambari.server.view.ViewURLStreamProvider.getConnectionAsCurrent(ViewURLStreamProvider.java:180)\n\tat org.apache.ambari.view.tez.utils.ProxyHelper.getResponse(ProxyHelper.java:61)\n\tat org.apache.ambari.view.tez.rest.BaseProxyResource.getData(BaseProxyResource.java:64)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)\n\tat com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)\n\tat org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)\n\tat org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:257)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.apache.ambari.server.security.authorization.jwt.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:96)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)\n\tat org.apache.ambari.server.security.authentication.AmbariAuthenticationFilter.doFilter(AmbariAuthenticationFilter.java:88)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:91)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)\n\tat org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)\n\tat org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)\n\tat org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)\n\tat org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.apache.ambari.server.view.AmbariViewsMDCLoggingFilter.doFilter(AmbariViewsMDCLoggingFilter.java:54)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.apache.ambari.server.view.ViewThrottleFilter.doFilter(ViewThrottleFilter.java:161)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:109)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:109)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)\n\tat org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)\n\tat org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:212)\n\tat org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:201)\n\tat org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:150)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:370)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973)\n\tat org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035)\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)\n\tat org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)\n\tat org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)\n\tat org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\n\tat java.lang.Thread.run(Thread.java:745)\n" } Request: { "adapterName": "dag", "url": "http://hostname2:8080/api/v1/views/TEZ/versions/0.7.0.2.5.3.0-136/instances/TEZ_CLUSTER_INSTANCE/resources/atsproxy/ws/v1/timeline/TEZ_DAG_ID", "responseHeaders": { "User": "admin", "Server": "Jetty(8.1.19.v20160209)", "X-Frame-Options": "SAMEORIGIN", "Content-Length": "11137", "X-XSS-Protection": "1; mode=block", "Content-Type": "application/json" }, "queryParams": { "limit": 11 }, "namespace": "dags" } Stack: Error: Adapter operation failed at ember$data$lib$adapters$errors$AdapterError.EmberError (http://hostname28080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:24586:21) at ember$data$lib$adapters$errors$AdapterError (http://hostname2:8080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:80222:50) at Class.handleResponse (http://hostname2:8080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:81517:16) at Class.hash.error (http://hostname2:8080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:81597:33) at fire (http://hostname2:8080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:3320:30) at Object.fireWith [as rejectWith] (http://hostname2:8080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:3432:7) at done (http://hostname2:8080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:8487:14) at XMLHttpRequest.<anonymous> (http://hostname2:8080/views/TEZ/0.7.0.2.5.3.0-136/TEZ_CLUSTER_INSTANCE/assets/vendor.js:8826:9)

Online	Offline
Last Visited	‎04-22-2024 12:23 PM

Member Since	‎03-20-2017 04:58 AM
Last Visited	‎04-22-2024 12:23 PM
Posts	6
Kudos received	1

Cloudera Community

Re: How to alert when datanode has a disk/volume f...

Re: Tez UI not coming up, failing with spnego auth

Tez UI not coming up, failing with spnego auth