@nonames Go to one of the hosts and look through the /var/log/cloudera-scm-agent/cloudera-scm-agent.log file for any exceptions talking to the CM. Also go to the CM->Hosts tab and check that the agents are heartbeating (under 15 secs each interval). If the agents are in bad health and not talking to CM then there will no way for the agents to know to fetch a parcel from CM so you must resolve the health issues with the agents to the CM.
... View more
Change the directories below for Service Monitor since the procedure is the same as for the Host Monitor. You can salvage the contents of the Host Monitor by using the LDBStoreTool Java Class to repair the corrupted LDB: Make sure the Host Monitor is stopped completely (it should be since it is unable to open this LDB). Backup the /var/lib/cloudera-host-monitor directory with tar or cp. Run the LDBStoreTool Java class to try and bring the corrupt database to a consistent state (please adjust the directory to the one reported in the exception): java -cp "/usr/share/cmf/lib/*" com.cloudera.cmon.tstore.leveldb.tool.LDBStoreTool repair --directory /var/lib/cloudera-host-monitor/subject_record/subject_ts/partitions/subject_ts_2017-10-30T18:03:04.415Z
[ main] log INFO Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[ main] CMONConfiguration INFO Config: jar:file:/usr/share/cmf/common_jars/firehose-5.12.1.jar!/cmon.conf
[ main] ConfigUtil WARN Could not find configuration file cmon-cm-auth.conf
[ main] LDBResourceManager INFO Max file descriptors: 4096
[ main] LDBResourceManager INFO Setting maximum open fds to: 2048
Running repair command
Start the Host Monitor and it should start now. If the LDBStoreTool Java class is unable to repair the corrupt LDB then you will have to purge the /var/lib/cloudera-host-monitor directory similar to steps noted above by Michalis.
... View more