Reply
Champion
Posts: 768
Registered: ‎05-16-2016

Re: Failed to connect to previous supervisor

@darkdante

 

The commuity had some kernel version compatability issue a month back if I remember 

You should be  good with 5.12 we are using 5.12 guess what HUE interface has some new functionalties and many more 

New Contributor
Posts: 2
Registered: ‎07-18-2017

Re: Failed to connect to previous supervisor

[ Edited ]

I've experiened the same problem on Ubuntu 16.04 LTS with Cloudera CDH 5.12

 

Out of 5 nodes, 4 upgraded with no problems, while the 5th has mentioned issue. All instances were working fine on CDH 5.11

 

One clue is the fact that supervisord directories were created but are empty

 

Is there a way to enable additional logging for supervisord during installation?

Champion
Posts: 768
Registered: ‎05-16-2016

Re: Failed to connect to previous supervisor

see if you have multiple supervisord runining 

ps aux   |   grep supervisor

share the logs  to see whats it is complain about supervisord 

 

Champion
Posts: 768
Registered: ‎05-16-2016

Re: Failed to connect to previous supervisor

@darkdante  were you able to receive heartbeat from agent.

 

Explorer
Posts: 6
Registered: ‎07-06-2017

Re: Failed to connect to previous supervisor

Hi csguna,

 

Here's the cloudera-scm-agent.log

 

 

[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Created /run/cloudera-scm-agent/process
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/process to 0751
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Created /run/cloudera-scm-agent/supervisor
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Created /run/cloudera-scm-agent/flood
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Chowning /run/cloudera-scm-agent/flood to cloudera-scm (988) cloudera-scm (983)
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/flood to 0751
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Created /run/cloudera-scm-agent/supervisor/include
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
[18/Jul/2017 17:38:20 +0000] 15294 MainThread agent        ERROR    Failed to connect to previous supervisor.
Traceback (most recent call last):
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor
    self.configure_supervisor_clients()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients
    supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")])
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize
    Options.realize(self, *arg, **kw)
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize
    self.process_config()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config
    self.process_config_file(do_usage)
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file
    self.usage(str(msg))
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage
    self.exit(2)
SystemExit: 2
[18/Jul/2017 17:38:21 +0000] 15294 Dummy-1 daemonize    WARNING  Stopping daemon.
[18/Jul/2017 17:38:21 +0000] 15294 Dummy-1 agent        INFO     Stopping agent...
[18/Jul/2017 17:38:21 +0000] 15294 Dummy-1 agent        INFO     No extant cgroups; unmounting any cgroup roots

and this is the result of ps aux | grep supervisor

 

root      15823  0.0  0.0 112644   972 pts/1    S+   17:57   0:00 grep --color=auto supervisor

As of today, it still outputs heartbeat failure.

Champion
Posts: 768
Registered: ‎05-16-2016

Re: Failed to connect to previous supervisor

bear with me but did you change the /etc/sysconfig/network file in all the nodes 1 2 3 4 as mentioned earlier ? 

did you restart the network  in your os ? 

 

New Contributor
Posts: 2
Registered: ‎07-18-2017

Re: Failed to connect to previous supervisor

At the end what worked for me was:

1) kernel update to the newest version

2) removal of all the alternatives to java and javac, e.g. https://askubuntu.com/questions/613016/removing-oracle-jdk-and-re-configuring-update-alternatives

Highlighted
Explorer
Posts: 6
Registered: ‎07-06-2017

Re: Failed to connect to previous supervisor

This is what I did instead. I followed Path B and downloaded 5.11.1 version instead. Solved all of my problems.

Explorer
Posts: 6
Registered: ‎08-16-2017

Re: Failed to connect to previous supervisor

I still have the same problem with slight modifications :)

 

I was able to roll back the agent to 5.11, but the rest of cluster were successfully upgrated to 5.12.

So, the 5.11 agent works fine, but the node is not visible on the Cloudera Manager 5.12.

 

New Contributor
Posts: 2
Registered: ‎09-06-2017

Re: Failed to connect to previous supervisor

[ Edited ]

Using 5.11 instead of 5.13 did not work for me. Neither did work the change in /etc/sysconfig/network:

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=poweredge

Announcements