Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Cloudera Community

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.
Solved Go to solution

Error starting Resource Manager - http server

Labels:
avatar
author-rank tomazhauzer
Frequent Visitor

Created on ‎07-16-2015 06:37 AM - edited ‎09-16-2022 02:34 AM

Hi,

 

when i try to start the Resource Manager i'm getting a error that i pasted at the end of the message.

Using Debian 8 in VirtualBox.

 

I was following this:

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_cdh5_install.htm...

 

Any suggestions what i'm missing here? Have no idea what connection does the http server have with resource manager.

 

This all the data i could collect about this:

 

root@debian8Node2:~# service hadoop-yarn-resourcemanager restart
Job for hadoop-yarn-resourcemanager.service failed. See 'systemctl status hadoop-yarn-resourcemanager.service' and 'journalctl -xn' for details.
root@debian8Node2:~# systemctl status hadoop-yarn-resourcemanager.service
● hadoop-yarn-resourcemanager.service - LSB: Hadoop resourcemanager
   Loaded: loaded (/etc/init.d/hadoop-yarn-resourcemanager)
   Active: failed (Result: exit-code) since Thu 2015-07-16 12:14:41 BST; 13s ago
  Process: 1094 ExecStop=/etc/init.d/hadoop-yarn-resourcemanager stop (code=exited, status=0/SUCCESS)
  Process: 1144 ExecStart=/etc/init.d/hadoop-yarn-resourcemanager start (code=exited, status=1/FAILURE)

Jul 16 12:14:35 debian8Node2 su[1176]: Successful su for yarn by root
Jul 16 12:14:35 debian8Node2 su[1176]: + ??? root:yarn
Jul 16 12:14:35 debian8Node2 su[1176]: pam_unix(su:session): session opened for user yarn by (uid=0)
Jul 16 12:14:35 debian8Node2 hadoop-yarn-resourcemanager[1144]: starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-debian8Node2.out
Jul 16 12:14:41 debian8Node2 hadoop-yarn-resourcemanager[1144]: Failed to start Hadoop resourcemanager. Return value: 1 ... failed!
Jul 16 12:14:41 debian8Node2 systemd[1]: hadoop-yarn-resourcemanager.service: control process exited, code=exited status=1
Jul 16 12:14:41 debian8Node2 systemd[1]: Failed to start LSB: Hadoop resourcemanager.
Jul 16 12:14:41 debian8Node2 systemd[1]: Unit hadoop-yarn-resourcemanager.service entered failed state.
root@debian8Node2:~# journalctl -xn
-- Logs begin at Thu 2015-07-16 07:45:17 BST, end at Thu 2015-07-16 12:17:01 BST. --
Jul 16 12:14:35 debian8Node2 su[1176]: pam_unix(su:session): session opened for user yarn by (uid=0)
Jul 16 12:14:35 debian8Node2 hadoop-yarn-resourcemanager[1144]: starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-debian8Node2.out
Jul 16 12:14:36 debian8Node2 su[1176]: pam_unix(su:session): session closed for user yarn
Jul 16 12:14:41 debian8Node2 hadoop-yarn-resourcemanager[1144]: Failed to start Hadoop resourcemanager. Return value: 1 ... failed!
Jul 16 12:14:41 debian8Node2 systemd[1]: hadoop-yarn-resourcemanager.service: control process exited, code=exited status=1
Jul 16 12:14:41 debian8Node2 systemd[1]: Failed to start LSB: Hadoop resourcemanager.
-- Subject: Unit hadoop-yarn-resourcemanager.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit hadoop-yarn-resourcemanager.service has failed.
--
-- The result is failed.
Jul 16 12:14:41 debian8Node2 systemd[1]: Unit hadoop-yarn-resourcemanager.service entered failed state.
Jul 16 12:17:01 debian8Node2 CRON[1426]: pam_unix(cron:session): session opened for user root by (uid=0)
Jul 16 12:17:01 debian8Node2 CRON[1427]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jul 16 12:17:01 debian8Node2 CRON[1426]: pam_unix(cron:session): session closed for user root

 

 

Ths from the /var/log/hadoop-yarn/yarn-yarn-resourcemanager-debian8Node2.log, pasted only the exception part

2015-07-16 12:14:36,927 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2015-07-16 12:14:36,933 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8088
2015-07-16 12:14:37,287 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager failed in state STARTED; cause: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:278)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:983)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1083)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1215)
Caused by: java.io.IOException: Problem in starting http server. Server handlers failed
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:841)
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
        ... 4 more

28,462 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank Harsh J author-rank
Mentor

Created ‎07-20-2015 02:37 AM

Thank you - it appears the logs are masking the thread failures at the INFO level. Could you change your logging config into DEBUG for the RM and retry to see if you notice any exceptions pertaining to the handler threads starting up?

You can switch it in the log4j.properties file under /etc/hadoop/conf/ on the RM host.

I see another email in the cdh-user@ group where something similar was observed for a user on Ubuntu, but that happened to them cause they had a mix-up with the zookeeper package provided by Ubuntu's own repository vs. CDH ones being installed. I am not sure if your Debian repository carries a zookeeper package as well but you should probably check via dpkg/etc. to ensure its a CDH version package and not a Debian-default one.

View solution in original post

28,425 Views
0 Kudos
9 REPLIES 9

avatar
author-rank Harsh J author-rank
Mentor

Created ‎07-19-2015 04:59 AM

Could you attach your full RM log here, via pastebin.com or such service, and also the .out file you will find alongside the .log file?

The abort's happening cause we're unable to run some vital threads within the RM. From the short log excerpt though, the reason is not very clear.
28,436 Views
0 Kudos

avatar
author-rank tomazhauzer
Frequent Visitor

Created ‎07-20-2015 01:16 AM

This are todays logs when i started it up. Hope it helps.

 

The .log file:

RM Log file

 

.out file:

.out file

28,429 Views
0 Kudos

avatar
author-rank Harsh J author-rank
Mentor

Created ‎07-20-2015 02:37 AM

Thank you - it appears the logs are masking the thread failures at the INFO level. Could you change your logging config into DEBUG for the RM and retry to see if you notice any exceptions pertaining to the handler threads starting up?

You can switch it in the log4j.properties file under /etc/hadoop/conf/ on the RM host.

I see another email in the cdh-user@ group where something similar was observed for a user on Ubuntu, but that happened to them cause they had a mix-up with the zookeeper package provided by Ubuntu's own repository vs. CDH ones being installed. I am not sure if your Debian repository carries a zookeeper package as well but you should probably check via dpkg/etc. to ensure its a CDH version package and not a Debian-default one.
28,426 Views
0 Kudos

avatar
author-rank tomazhauzer
Frequent Visitor

Created ‎07-21-2015 05:41 AM

Changed the log level, but the log contained nothing more then the log i linked before.

 

But the resource manager started when i purged the packages and made sure they are pulled from cdh5 repository.

 

Thanks for the suggestion.

28,415 Views
0 Kudos

avatar
author-rank jarnix
New Contributor

Created ‎11-19-2015 04:52 AM

Hi,

Can you explain how did you purge the packages ?

Thanks
27,628 Views
0 Kudos

avatar
author-rank jarnix
New Contributor

Created ‎11-19-2015 08:54 AM

I noticed that I was using the jdk 1.8, so I reinstalled the 1.7, but still I have an error at the start of the resource manager :

  

service hadoop-yarn-resourcemanager start

 

Job for hadoop-yarn-resourcemanager.service failed. See 'systemctl status hadoop-yarn-resourcemanager.service' and 'journalctl -xn' for details.

 

 

And in the logfile I cannot see why it fails :

 

2015-11-19 17:52:34,827 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8088
2015-11-19 17:52:35,480 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager failed in state STARTED; cause: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:278)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:985)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1085)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1217)
Caused by: java.io.IOException: Problem in starting http server. Server handlers failed
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:893)
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
        ... 4 more

 

27,614 Views
0 Kudos

avatar
author-rank Aphex
Visitor

Created ‎09-04-2017 09:22 PM

Hi
Can you explain how you purge them in thread, please
Thanks

23,082 Views
0 Kudos

avatar
author-rank wanderer
New Member

Created ‎09-21-2017 12:24 AM

Reading through this blog helped me a lot: http://blog.cloudera.com/blog/2014/11/guidelines-for-installing-cdh-packages-on-unsupported-operatin...

 

Specially understanding that this can be a package mix/match issue because of trying to run YARN on a unsupported version (Ubuntu 16.04).

 

I removed package and reinstalled like this:

 

$> sudo apt-get remove zookeeper 

 

this will remove a bunch of stuff along with zookeeper

 

$> sudo apt-get install hadoop-yarn-resourcemanager

 

this will reinstall resource manager for you.

 

Hope this helps!

22,867 Views
0 Kudos

avatar
author-rank Sudee
Explorer

Created ‎04-12-2016 08:59 PM

Hi,

 

   Could you please explain how to purge the packages and start the RM again.

 

Thanks

26,598 Views
0 Kudos
Announcements
What's New @ Cloudera
Announcing Cloudera Data Flow 3.2 for Cloudera on Cloud
Community Announcements
Community Upgrade Announcement
What's New @ Cloudera
Announcing Cloudera Flow Management Operator for Kubernetes ...
What's New @ Cloudera
Announcing Cloudera Streams Messaging Operator for Kubernete...
Community Announcements
July 2026 Community Highlights
View More Announcements