Reply
Contributor
Posts: 90
Registered: ‎11-12-2015

Cloudera manager services won't start after machine reboot

Hello,

 

I had a running Cloudera culster with all the agents OK. But recently I had to reboot the cloudera-scm-server/agent machine, and other agent machine. But now the Cloudera manager services (Host Monitor, Service Monitor, Alert Publisher and Event Server) won't start and gave me this error:

 

Command aborted because of exception: Command timed-out after 150 seconds

This is the command that tried to execute:

 

  • mgmt/mgmt.sh ["firehose","--pipeline-type","HOST_MONITORING"]

 

This is a screenshot of the error:

cloudera-manager-service-error.png

 

Also the logs that I have from the Firehose are from before the restart. I was looking in al the cloudera-scm-* logs in order to find the error, and I found this in cloudera-scm-agent logs:

 

 

 56839 MainThread agent        ERROR    Failed to configure inotify. Parcel repository will not auto-refresh.
Traceback (most recent call last):
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.8.1-py2.7.egg/cmf/agent.py", line 791, in _init_after_first_heartbeat_response
    self.inotify = self.repo.configure_inotify()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.8.1-py2.7.egg/cmf/parcel.py", line 415, in configure_inotify
    wm = pyinotify.WatchManager()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/pyinotify-0.9.3-py2.7.egg/pyinotify.py", line 1706, in __init__
    raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)

 

I don't know if this is the root of the error. But this is all I can find.

 

Regards.

 

 

 

Announcements