Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Error starting Resource Manager - http server

Labels:
avatar
author-rank tomazhauzer
New Contributor

Created on ‎07-16-2015 06:37 AM - edited ‎09-16-2022 02:34 AM

Hi,

 

when i try to start the Resource Manager i'm getting a error that i pasted at the end of the message.

Using Debian 8 in VirtualBox.

 

I was following this:

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_cdh5_install.htm...

 

Any suggestions what i'm missing here? Have no idea what connection does the http server have with resource manager.

 

This all the data i could collect about this:

 

root@debian8Node2:~# service hadoop-yarn-resourcemanager restart
Job for hadoop-yarn-resourcemanager.service failed. See 'systemctl status hadoop-yarn-resourcemanager.service' and 'journalctl -xn' for details.
root@debian8Node2:~# systemctl status hadoop-yarn-resourcemanager.service
● hadoop-yarn-resourcemanager.service - LSB: Hadoop resourcemanager
   Loaded: loaded (/etc/init.d/hadoop-yarn-resourcemanager)
   Active: failed (Result: exit-code) since Thu 2015-07-16 12:14:41 BST; 13s ago
  Process: 1094 ExecStop=/etc/init.d/hadoop-yarn-resourcemanager stop (code=exited, status=0/SUCCESS)
  Process: 1144 ExecStart=/etc/init.d/hadoop-yarn-resourcemanager start (code=exited, status=1/FAILURE)

Jul 16 12:14:35 debian8Node2 su[1176]: Successful su for yarn by root
Jul 16 12:14:35 debian8Node2 su[1176]: + ??? root:yarn
Jul 16 12:14:35 debian8Node2 su[1176]: pam_unix(su:session): session opened for user yarn by (uid=0)
Jul 16 12:14:35 debian8Node2 hadoop-yarn-resourcemanager[1144]: starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-debian8Node2.out
Jul 16 12:14:41 debian8Node2 hadoop-yarn-resourcemanager[1144]: Failed to start Hadoop resourcemanager. Return value: 1 ... failed!
Jul 16 12:14:41 debian8Node2 systemd[1]: hadoop-yarn-resourcemanager.service: control process exited, code=exited status=1
Jul 16 12:14:41 debian8Node2 systemd[1]: Failed to start LSB: Hadoop resourcemanager.
Jul 16 12:14:41 debian8Node2 systemd[1]: Unit hadoop-yarn-resourcemanager.service entered failed state.
root@debian8Node2:~# journalctl -xn
-- Logs begin at Thu 2015-07-16 07:45:17 BST, end at Thu 2015-07-16 12:17:01 BST. --
Jul 16 12:14:35 debian8Node2 su[1176]: pam_unix(su:session): session opened for user yarn by (uid=0)
Jul 16 12:14:35 debian8Node2 hadoop-yarn-resourcemanager[1144]: starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-debian8Node2.out
Jul 16 12:14:36 debian8Node2 su[1176]: pam_unix(su:session): session closed for user yarn
Jul 16 12:14:41 debian8Node2 hadoop-yarn-resourcemanager[1144]: Failed to start Hadoop resourcemanager. Return value: 1 ... failed!
Jul 16 12:14:41 debian8Node2 systemd[1]: hadoop-yarn-resourcemanager.service: control process exited, code=exited status=1
Jul 16 12:14:41 debian8Node2 systemd[1]: Failed to start LSB: Hadoop resourcemanager.
-- Subject: Unit hadoop-yarn-resourcemanager.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit hadoop-yarn-resourcemanager.service has failed.
--
-- The result is failed.
Jul 16 12:14:41 debian8Node2 systemd[1]: Unit hadoop-yarn-resourcemanager.service entered failed state.
Jul 16 12:17:01 debian8Node2 CRON[1426]: pam_unix(cron:session): session opened for user root by (uid=0)
Jul 16 12:17:01 debian8Node2 CRON[1427]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jul 16 12:17:01 debian8Node2 CRON[1426]: pam_unix(cron:session): session closed for user root

 

 

Ths from the /var/log/hadoop-yarn/yarn-yarn-resourcemanager-debian8Node2.log, pasted only the exception part

2015-07-16 12:14:36,927 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2015-07-16 12:14:36,933 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8088
2015-07-16 12:14:37,287 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager failed in state STARTED; cause: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:278)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:983)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1083)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1215)
Caused by: java.io.IOException: Problem in starting http server. Server handlers failed
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:841)
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
        ... 4 more

24,061 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank Harsh J author-rank
Mentor

Created ‎07-20-2015 02:37 AM

Thank you - it appears the logs are masking the thread failures at the INFO level. Could you change your logging config into DEBUG for the RM and retry to see if you notice any exceptions pertaining to the handler threads starting up?

You can switch it in the log4j.properties file under /etc/hadoop/conf/ on the RM host.

I see another email in the cdh-user@ group where something similar was observed for a user on Ubuntu, but that happened to them cause they had a mix-up with the zookeeper package provided by Ubuntu's own repository vs. CDH ones being installed. I am not sure if your Debian repository carries a zookeeper package as well but you should probably check via dpkg/etc. to ensure its a CDH version package and not a Debian-default one.

View solution in original post

24,022 Views
0 Kudos
9 REPLIES 9

avatar
author-rank Harsh J author-rank
Mentor

Created ‎07-19-2015 04:59 AM

Could you attach your full RM log here, via pastebin.com or such service, and also the .out file you will find alongside the .log file?

The abort's happening cause we're unable to run some vital threads within the RM. From the short log excerpt though, the reason is not very clear.
24,035 Views
0 Kudos

avatar
author-rank tomazhauzer
New Contributor

Created ‎07-20-2015 01:16 AM

This are todays logs when i started it up. Hope it helps.

 

The .log file:

RM Log file

 

.out file:

.out file

24,026 Views
0 Kudos

avatar
author-rank Harsh J author-rank
Mentor

Created ‎07-20-2015 02:37 AM

Thank you - it appears the logs are masking the thread failures at the INFO level. Could you change your logging config into DEBUG for the RM and retry to see if you notice any exceptions pertaining to the handler threads starting up?

You can switch it in the log4j.properties file under /etc/hadoop/conf/ on the RM host.

I see another email in the cdh-user@ group where something similar was observed for a user on Ubuntu, but that happened to them cause they had a mix-up with the zookeeper package provided by Ubuntu's own repository vs. CDH ones being installed. I am not sure if your Debian repository carries a zookeeper package as well but you should probably check via dpkg/etc. to ensure its a CDH version package and not a Debian-default one.
24,023 Views
0 Kudos

avatar
author-rank tomazhauzer
New Contributor

Created ‎07-21-2015 05:41 AM

Changed the log level, but the log contained nothing more then the log i linked before.

 

But the resource manager started when i purged the packages and made sure they are pulled from cdh5 repository.

 

Thanks for the suggestion.

24,012 Views
0 Kudos

avatar
author-rank jarnix
New Contributor

Created ‎11-19-2015 04:52 AM

Hi,

Can you explain how did you purge the packages ?

Thanks
23,226 Views
0 Kudos

avatar
author-rank jarnix
New Contributor

Created ‎11-19-2015 08:54 AM

I noticed that I was using the jdk 1.8, so I reinstalled the 1.7, but still I have an error at the start of the resource manager :

  

service hadoop-yarn-resourcemanager start

 

Job for hadoop-yarn-resourcemanager.service failed. See 'systemctl status hadoop-yarn-resourcemanager.service' and 'journalctl -xn' for details.

 

 

And in the logfile I cannot see why it fails :

 

2015-11-19 17:52:34,827 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8088
2015-11-19 17:52:35,480 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager failed in state STARTED; cause: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:278)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:985)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1085)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1217)
Caused by: java.io.IOException: Problem in starting http server. Server handlers failed
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:893)
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
        ... 4 more

 

23,213 Views
0 Kudos

avatar
author-rank Aphex
New Contributor

Created ‎09-04-2017 09:22 PM

Hi
Can you explain how you purge them in thread, please
Thanks

18,679 Views
0 Kudos

avatar
author-rank wanderer
New Contributor

Created ‎09-21-2017 12:24 AM

Reading through this blog helped me a lot: http://blog.cloudera.com/blog/2014/11/guidelines-for-installing-cdh-packages-on-unsupported-operatin...

 

Specially understanding that this can be a package mix/match issue because of trying to run YARN on a unsupported version (Ubuntu 16.04).

 

I removed package and reinstalled like this:

 

$> sudo apt-get remove zookeeper 

 

this will remove a bunch of stuff along with zookeeper

 

$> sudo apt-get install hadoop-yarn-resourcemanager

 

this will reinstall resource manager for you.

 

Hope this helps!

18,464 Views
0 Kudos

avatar
author-rank Sudee
Explorer

Created ‎04-12-2016 08:59 PM

Hi,

 

   Could you please explain how to purge the packages and start the RM again.

 

Thanks

22,194 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements