CDH5.5.1 failed to start history server

threecuptea — Fri, 16 Sep 2022 09:55:27 GMT

Hi:
I used to have CDH5.4.7. I could start all hadoop yarn servers O.K. in pseudo distributed mode. I just upgraded to 5.5 today. I found that I couldn't start history server now. The error is like the followings. I checked 'netstat'. Port 19888 is not binded to any process. If I changed mapreduce.jobhistory.webapp.address in mapred-site.xml to 19088. I would get the similar error with Port in use: 19088. Please advise how I can resolve this issue or is this a bug.Thanks.

6/01/02 15:15:52 INFO service.AbstractService: Service HistoryClientService failed in state STARTED; cause: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:278)
at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService.initializeWebApp(HistoryClientService.java:156)
at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService.serviceStart(HistoryClientService.java:121)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceStart(JobHistoryServer.java:197)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:223)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:232)
Caused by: java.net.BindException: Port in use: ununtu:19888
at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:944)
at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:880)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
... 8 more
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:939)
... 10 more

Re: CDH5.5.1 failed to start history server

valtri — Mon, 04 Jan 2016 21:41:18 GMT

Just a wild guess: is IPv6 disabled?

-Djava.net.preferIPv4Stack=true in the HS process arguments
or no IPv6 addresses on the interfaces

You could try to set this just to be sure (but I have this set to "0" and it is working fine anyway):

sysctl net.ipv6.bindv6only = 1

Re: CDH5.5.1 failed to start history server

threecuptea — Tue, 05 Jan 2016 00:07:09 GMT

I didn't make any changes. Somehow it is working now.

Re: CDH5.5.1 failed to start history server

rschmidt13 — Tue, 28 Feb 2017 15:53:18 GMT

I had the same issue. The solution was to start the history server first. This means before the other cdh services like hdfs and yarn are started.

question Re: CDH5.5.1 failed to start history server in Archives of Support Questions (Read Only)

CDH5.5.1 failed to start history server

Re: CDH5.5.1 failed to start history server

Re: CDH5.5.1 failed to start history server

Re: CDH5.5.1 failed to start history server