Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Cloudera manager services won't start after machine reboot

Cloudera manager services won't start after machine reboot

Contributor

Hello,

 

I had a running Cloudera culster with all the agents OK. But recently I had to reboot the cloudera-scm-server/agent machine, and other agent machine. But now the Cloudera manager services (Host Monitor, Service Monitor, Alert Publisher and Event Server) won't start and gave me this error:

 

Command aborted because of exception: Command timed-out after 150 seconds

This is the command that tried to execute:

 

  • mgmt/mgmt.sh ["firehose","--pipeline-type","HOST_MONITORING"]

 

This is a screenshot of the error:

cloudera-manager-service-error.png

 

Also the logs that I have from the Firehose are from before the restart. I was looking in al the cloudera-scm-* logs in order to find the error, and I found this in cloudera-scm-agent logs:

 

 

 56839 MainThread agent        ERROR    Failed to configure inotify. Parcel repository will not auto-refresh.
Traceback (most recent call last):
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.8.1-py2.7.egg/cmf/agent.py", line 791, in _init_after_first_heartbeat_response
    self.inotify = self.repo.configure_inotify()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.8.1-py2.7.egg/cmf/parcel.py", line 415, in configure_inotify
    wm = pyinotify.WatchManager()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/pyinotify-0.9.3-py2.7.egg/pyinotify.py", line 1706, in __init__
    raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)

 

I don't know if this is the root of the error. But this is all I can find.

 

Regards.