Reply
Expert Contributor
Posts: 162
Registered: ‎07-29-2013

CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

How can we fix it?

[07/Nov/2014 20:17:15 +0000] 10547 MainThread _cplogging ERROR [07/Nov/2014:20:17:15] ENGINE Error in 'start' listener <bound method Server.start of <cherrypy._cpserver.Server object at 0x7f06acb67e90>>
Traceback (most recent call last):
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/process/wspbus.py", line 197, in publish
output.append(listener(*args, **kwargs))
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/_cpserver.py", line 151, in start
ServerAdapter.start(self)
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/process/servers.py", line 167, in start
wait_for_free_port(*self.bind_addr)
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/process/servers.py", line 410, in wait_for_free_port
raise IOError("Port %r not free on %r" % (port, host))
IOError: Port 9000 not free on 'napoleon01.desert.ru'

[07/Nov/2014 20:17:15 +0000] 10547 MainThread _cplogging ERROR [07/Nov/2014:20:17:15] ENGINE Shutting down due to error in start listener:
Traceback (most recent call last):
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/process/wspbus.py", line 235, in start
self.publish('start')
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/process/wspbus.py", line 215, in publish
raise exc
ChannelFailures: IOError("Port 9000 not free on 'napoleon01.desert.ru'",)

Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

One more from out:

 

/var/log/cloudera-scm-agent/cloudera-scm-agent.out
Traceback (most recent call last):
File "/usr/lib/cmf/agent/src/cmf/agent.py", line 8, in <module>
import avro.ipc
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/avro-1.6.3-py2.7.egg/avro/ipc.py", line 19, in <module>
import httplib
File "/usr/lib/python2.7/httplib.py", line 79, in <module>
import mimetools
File "/usr/lib/python2.7/mimetools.py", line 6, in <module>
import tempfile
File "/usr/lib/python2.7/tempfile.py", line 32, in <module>
import io as _io
File "/usr/lib/python2.7/io.py", line 51, in <module>
import _io

Cloudera Employee
Posts: 225
Registered: ‎09-23-2013

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

What Distribution and version of linux is this on?  Was this an upgrade or new install?

Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

Ubuntu Server 14.04

upgrade from 5.1.3 to 5.2

 

The other problem is (related or non related???) I can't add chart from charts library in CM UI.

I select charts and click "add" then I get "query not specified". There is nothing useful in CM server log...

New Contributor
Posts: 3
Registered: ‎12-08-2014

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

Did anyone find any resolution to this ? We are facing exactly the same issue .

Cloudera Employee
Posts: 225
Registered: ‎09-23-2013

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

It appeared that there was something listening or connecting to port 9000, per the original logging messages:

 

"Port 9000 not free on..."

 

If possible try the following to identify what pid/program is using port 9000

 

# netstat -tulpn 

 

It will give pid/program name to help track what is on 9000 when you see this.   might be the services did not shut down properly/are hung.  Then get detailed process information on the pid that shows up listening.

 

 

 

 

Highlighted
New Contributor
Posts: 3
Registered: ‎12-08-2014

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

Nothing was listening on 9000 or 9001.

[root@noividap2085 ~]# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:2222 0.0.0.0:* LISTEN 2066/sshd
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1770/rpcbind
tcp 0 0 0.0.0.0:2544 0.0.0.0:* LISTEN 2287/zmd
tcp 0 0 0.0.0.0:1556 0.0.0.0:* LISTEN 1924/pbx_exchange
tcp 0 0 127.0.0.1:1557 0.0.0.0:* LISTEN 1924/pbx_exchange
tcp 0 0 0.0.0.0:13782 0.0.0.0:* LISTEN 2106/bpcd
tcp 0 0 127.0.0.1:54391 0.0.0.0:* LISTEN 1924/pbx_exchange
tcp 0 0 127.0.0.1:19001 0.0.0.0:* LISTEN 31564/python
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 2229/master
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 32107/sshd
tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 32225/sshd
tcp 0 0 127.0.0.1:6012 0.0.0.0:* LISTEN 465/sshd
tcp 0 0 0.0.0.0:13724 0.0.0.0:* LISTEN 2102/vnetd
tcp 0 0 0.0.0.0:52093 0.0.0.0:* LISTEN 1790/rpc.statd
tcp 0 0 0.0.0.0:5950 0.0.0.0:* LISTEN 2076/xinetd
tcp 0 0 0.0.0.0:5951 0.0.0.0:* LISTEN 2076/xinetd
udp 0 0 0.0.0.0:111 0.0.0.0:* 1770/rpcbind
udp 0 0 10.230.68.23:123 0.0.0.0:* 2088/ntpd
udp 0 0 127.0.0.1:123 0.0.0.0:* 2088/ntpd
udp 0 0 0.0.0.0:123 0.0.0.0:* 2088/ntpd
udp 0 0 0.0.0.0:669 0.0.0.0:* 1770/rpcbind
udp 0 0 0.0.0.0:694 0.0.0.0:* 1790/rpc.statd
udp 0 0 0.0.0.0:49084 0.0.0.0:* 1790/rpc.statd
udp 0 0 0.0.0.0:52311 0.0.0.0:* 2308/BESClient

Explorer
Posts: 37
Registered: ‎05-03-2017

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

Having the same problem just right now but with the distribution CDH 5.10.0

Explorer
Posts: 28
Registered: ‎04-15-2018

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

In my case sshd is running on port:9000.

 

solomon@chinni:~$ sudo netstat -lnp | grep 9000
tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 26616/sshd
tcp6 0 0 :::9000 :::* LISTEN 26616/sshd

i)permitrootlogin is YES

ii)ufw is diabled

iii)hostname and ip address are correctly mentioned in the hostsfile

still same I'm unable to get heartbeat from agent.

 

Posts: 953
Topics: 1
Kudos: 226
Solutions: 121
Registered: ‎04-22-2014

Re: CDH 5.2 some problem with agent during start: ENGINE Error in 'start' listener

@solomonchinni,

 

While it would not be the cause of heartbeat failures, the agent's default listening port is 9000, so the agent will not function completely with sshd using that port.

 

You would have to change the port in /etc/cloudera-scm-agent/config.ini:

 

## It should not normally be necessary to modify these.
# Port that the CM agent should listen on.
# listening_port=9000

 

That said, there are many reasons heartbeats may fail.  I would recommend starting a new thread and list out:

 

- What you are trying to do

- What you observe that indicates there is a problem

- Any agent log messages and context that are relevant.

 

 

Announcements