Posts: 90
Registered: ‎11-12-2015

Cloudera manager services won't start after machine reboot



I had a running Cloudera culster with all the agents OK. But recently I had to reboot the cloudera-scm-server/agent machine, and other agent machine. But now the Cloudera manager services (Host Monitor, Service Monitor, Alert Publisher and Event Server) won't start and gave me this error:


Command aborted because of exception: Command timed-out after 150 seconds

This is the command that tried to execute:


  • mgmt/ ["firehose","--pipeline-type","HOST_MONITORING"]


This is a screenshot of the error:



Also the logs that I have from the Firehose are from before the restart. I was looking in al the cloudera-scm-* logs in order to find the error, and I found this in cloudera-scm-agent logs:



 56839 MainThread agent        ERROR    Failed to configure inotify. Parcel repository will not auto-refresh.
Traceback (most recent call last):
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.8.1-py2.7.egg/cmf/", line 791, in _init_after_first_heartbeat_response
    self.inotify = self.repo.configure_inotify()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.8.1-py2.7.egg/cmf/", line 415, in configure_inotify
    wm = pyinotify.WatchManager()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/pyinotify-0.9.3-py2.7.egg/", line 1706, in __init__
    raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)


I don't know if this is the root of the error. But this is all I can find.