About michael_boulter

michael_boulter · ‎07-19-2021

Dear Cloudera community I'm struggling to get Hive jobs to run on Tez on our kerberized cluster with HA RM. We're getting the following error, does anyone have any idea what the problem is? Kind regards Michael 2021-07-19 16:51:41,744 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |http.HttpServer2|: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2021-07-19 16:51:41,787 [ERROR] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |web.WebUIService|: Tez UI WebService failed to start. java.lang.NullPointerException at org.apache.hadoop.util.StringUtils.join(StringUtils.java:941) at org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer.initFilter(AmFilterInitializer.java:74) at org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:587) at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:537) at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:117) at org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:421) at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:340) at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:427) at org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:94) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194) at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1865) at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1886)

michael_boulter · ‎07-19-2021

Thanks! Before you replied I tried the following which also helped: yarn.scheduler.capacity.schedule-asynchronously.maximum-threads=2 yarn.scheduler.capacity.schedule-asynchronously.scheduling-interval-ms=100

michael_boulter · ‎05-11-2021

Hi We are periodically getting errors in our resource manager logs, which then cause no Spark jobs to be accepted until we restart the YARN ResourceManager. We are on HDP 3.1.4 and we have Kerberos enabled. The stack trace is below. Does anyone have ideas, please? 2021-05-08 04:15:59,407 ERROR yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(68)) - Thread Thread[Thread-20,5,main] threw an Exception. java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.ResourceCommitRequest.<init>(ResourceCommitRequest.java:101) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.createResourceCommitRequest(CapacityScheduler.java:2848) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.submitResourceCommitRequest(CapacityScheduler.java:2685) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainerOnSingleNode(CapacityScheduler.java:1558) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1684) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1436) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.schedule(CapacityScheduler.java:548) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$AsyncScheduleThread.run(CapacityScheduler.java:595)

michael_boulter · ‎12-22-2016

Out of curiosity, why is R/SparkR only provided through Livy in HDP 2.5 as it is provided as standard with the standalone Zeppelin package? Thanks.

Online	Offline
Last Visited	‎06-14-2022 06:08 AM

Member Since	‎12-22-2016 03:22 PM
Last Visited	‎06-14-2022 06:08 AM
Posts	5
Kudos received	1

Cloudera Community

Re: YARN ResourceManager CapacityScheduler ERROR

Tez UI WebService failed to start

Re: YARN ResourceManager CapacityScheduler ERROR

YARN ResourceManager CapacityScheduler ERROR

Re: No R interpreter in Zeppelin, HDP 2.5