Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH5.5.1 failed to start history server

Solved Go to solution

CDH5.5.1 failed to start history server

New Contributor

Hi:
I used to have CDH5.4.7.  I could start all hadoop yarn servers O.K. in pseudo distributed mode.  I just upgraded to 5.5 today.  I found that I couldn't start history server now.  The error is like the followings.   I checked 'netstat'.  Port 19888 is not binded to any process.  If I changed mapreduce.jobhistory.webapp.address in mapred-site.xml to 19088.   I would get the similar error with Port in use: 19088.  Please advise how I can resolve this issue or is this a bug.Thanks.

 

6/01/02 15:15:52 INFO service.AbstractService: Service HistoryClientService failed in state STARTED; cause: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:278)
at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService.initializeWebApp(HistoryClientService.java:156)
at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService.serviceStart(HistoryClientService.java:121)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceStart(JobHistoryServer.java:197)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:223)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:232)
Caused by: java.net.BindException: Port in use: ununtu:19888
at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:944)
at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:880)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
... 8 more
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:939)
... 10 more

1 ACCEPTED SOLUTION

Accepted Solutions

Re: CDH5.5.1 failed to start history server

New Contributor

I didn't make any changes. Somehow it is working now. 

3 REPLIES 3

Re: CDH5.5.1 failed to start history server

Contributor

Just a wild guess: is IPv6 disabled?

  • -Djava.net.preferIPv4Stack=true in the HS process arguments
  • or no IPv6 addresses on the interfaces

You could try to set this just to be sure (but I have this set to "0" and it is working fine anyway):

  •  sysctl net.ipv6.bindv6only = 1

 

Re: CDH5.5.1 failed to start history server

New Contributor

I didn't make any changes. Somehow it is working now. 

Re: CDH5.5.1 failed to start history server

New Contributor

I had the same issue. The solution was to start the history server first. This means before the other cdh services like hdfs and yarn are started.

Don't have an account?
Coming from Hortonworks? Activate your account here