Member since
04-26-2017
9
Posts
0
Kudos Received
0
Solutions
10-08-2020
05:50 AM
In My HDP 3.1.5 cluster, Tez is not following the reducer order in the VERTICES (shown in the first pic). I am interested in knowing how Tez decides the order and number of containers in a VERTICES. I can even see single-digit containers in a few VERTICES. Resources are free on the cluster but it has very few containers running.
... View more
Labels:
10-07-2020
04:28 AM
Its a service that's running, not a job. So avoid killing it.
... View more
03-19-2019
10:22 PM
2019-03-19 04:48:41,934 INFO server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(240)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2019-03-19 04:48:41,940 INFO server.KerberosAuthenticationHandler (KerberosAuthenticationHandler.java:init(158)) - Using keytab /etc/security/keytabs/spnego.service.keytab, for principal HTTP/ppc64le-rhel75-50g-1.novalocal@EXAMPLE.COM 2019-03-19 04:48:41,951 INFO server.AbstractConnector (AbstractConnector.java:doStart(278)) - Started ServerConnector@2c5529ab{HTTP/1.1,[http/1.1]}{ppc64le-rhel75-50g-1.novalocal:50070} 2019-03-19 04:48:41,960 INFO server.AbstractConnector (AbstractConnector.java:doStop(318)) - Stopped ServerConnector@2c5529ab{HTTP/1.1,[http/1.1]}{ppc64le-rhel75-50g-1.novalocal:50070} 2019-03-19 04:48:41,963 INFO handler.ContextHandler (ContextHandler.java:doStop(910)) - Stopped o.e.j.s.ServletContextHandler@77258e59{/logs,file:///var/log/hadoop/hdfs/,UNAVAILABLE} 2019-03-19 04:48:41,967 INFO impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(210)) - Stopping NameNode metrics system... 2019-03-19 04:48:41,968 INFO impl.MetricsSinkAdapter (MetricsSinkAdapter.java:publishMetricsFromQueue(141)) - timeline thread interrupted. 2019-03-19 04:48:41,969 INFO impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(216)) - NameNode metrics system stopped. 2019-03-19 04:48:41,970 INFO impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(607)) - NameNode metrics system shutdown complete. 2019-03-19 04:48:41,970 ERROR namenode.NameNode (NameNode.java:main(1715)) - Failed to start namenode. java.io.IOException: Problem starting http server at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1165) at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:177) at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:869) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:691) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:937) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:910) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1643) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1710) Caused by: javax.servlet.ServletException: Public key for signature validation must be provisioned. at org.apache.hadoop.security.authentication.server.JWTRedirectAuthenticationHandler.init(JWTRedirectAuthenticationHandler.java:125) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeAuthHandler(AuthenticationFilter.java:194) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:180) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:349) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.server.Server.start(Server.java:427) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) at org.eclipse.jetty.server.Server.doStart(Server.java:394) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1134) ... 7 more 2019-03-19 04:48:41,972 INFO timeline.HadoopTimelineMetricsSink (AbstractTimelineMetricsSink.java:getCurrentCollectorHost(291)) - No live collector to send metrics to. Metrics to be sent will be discarded. This message will be skipped for the next 20 times. 2019-03-19 04:48:41,973 INFO util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status 1: java.io.IOException: Problem starting http server 2019-03-19 04:48:41,977 INFO namenode.NameNode (LogAdapter.java:info(51)) - SHUTDOWN_MSG:
... View more
Labels:
- Labels:
-
Apache Hadoop
08-14-2018
10:59 AM
@Facundo Bianco Please check disk space on the master node
... View more