question Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors in Support Questions

CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

Johnny K — Wed, 30 Apr 2014 23:57:33 GMT

Hi,

I installed CDH5 using installation path A and everthing is working with the exception of the monitoring.

All services have "Unknown Health", charts have the error of "Unable to issue query: could not connect to the Host Monitor", the top of the home page has "Request to the Service Monitor failed. This may cause slow page responses". The last error when clicking into it returns a 404 error.

Looking at the logs I'm seeing the following warnings:

1 - WARN [1752771961@scm-web-7:cmf.StatusProvider@503] Failed to get status.

2 - WARN [1202997703@scm-web-1:tsquery.TimeSeriesQueryService@503] Could not find a HOST_MONITORING nozzle from SCM.

3 - WARN [44428231@scm-web-34:cmf.EventsController@503] Exception querying events com.cloudera.cmon.MgmtServiceLocatorException: There is no configured event store

I'd appreciate any insight.

Thanks - J

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

Harsh J — Sat, 03 May 2014 13:29:45 GMT

Can you post your entire Service and Host monitor logs here, or via pastebin links?

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

hd — Thu, 25 Sep 2014 07:27:25 GMT

this is entire Service and Host monitor logs

ServerLog

Re: CM5 Service Monitor Failed

GautamG — Thu, 25 Sep 2014 16:00:15 GMT

I don't have a QQ account. Please use a service like pastebin that doesn't
need us to sign in.

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

helianthus_lu — Thu, 13 Nov 2014 13:45:56 GMT

Does your CM work well? I met the same truble: 2014-11-13 21:39:24,485 WARN 460709139@scm-web-34:com.cloudera.server.cmf.tsquery.TimeSeriesQueryService: (107 skipped) Could not find a HOST_MONITORING nozzle from SCM. com.cloudera.cmon.MgmtServiceLocatorException: Could not find a HOST_MONITORING nozzle from SCM. at com.cloudera.cmon.MgmtServiceLocator.getNozzleIPC(MgmtServiceLocator.java:141) at com.cloudera.server.cmf.tsquery.NozzleRequest.(NozzleRequest.java:50) at com.cloudera.server.cmf.tsquery.TimeSeriesMultiRequest.(TimeSeriesMultiRequest.java:48) at com.cloudera.server.cmf.tsquery.TimeSeriesQueryService.queryTimeSeries(TimeSeriesQueryService.java:306) at com.cloudera.server.web.cmf.charts.TimeSeriesQueryController.queryTimeSeriesHelper(TimeSeriesQueryController.java:303) at com.cloudera.server.web.cmf.charts.TimeSeriesQueryController.queryTimeSeries(TimeSeriesQueryController.java:269) at sun.reflect.GeneratedMethodAccessor808.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:436) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:424) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:585) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) …… ----------------------------------------------------------------------------------------------------------------------------------------------- I really don't know why? If anyone can tell me why ,I would be grateful.

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

helianthus_lu — Thu, 13 Nov 2014 14:05:10 GMT

Hello,I saw an answer ,maybe It can help you: http://grokbase.com/t/cloudera/scm-users/136jahv5gv/cloudera-manager-reporting-could-not-find-a-host-monitoring-nozzle-from-scm

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

Jony — Wed, 07 Jan 2015 03:53:07 GMT

Hi, i have the same problem,did you have resolve it ? very thanks~!

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

HDFS — Fri, 30 Jan 2015 17:04:40 GMT

i got the same error,

Restart your Cloudera management services,it will get working.

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

lizhenmxcz — Thu, 10 Dec 2015 07:43:37 GMT

i user cm version 5.4.5, after i reboot the cluster, i also has the same problem. The admin console output "Request to the Service Monitor failed. This may cause slow page responses. View the status of the Service Monitor.", who can solve it

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

AZWally — Wed, 20 Apr 2016 20:34:13 GMT

This may sound like a silly solution and I'm sure will not fix everyones problem but I was able to get both my host & service monitors back up and running with a simple restart of the monitors.

This has worked for me in 2 diffent CDH managed environments.

The question now is what caused them to fail in the first place?

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

evrenyildiz — Wed, 27 Apr 2016 06:02:03 GMT

I've check the log file under /var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-hadoop1.log.out and found the error below

2016-04-26 14:43:27,069 INFO com.cloudera.cmon.tstore.leveldb.LDBUtils: Creating directory /var/lib/cloudera-service-monitor/ts/ts_entity_metadata
2016-04-26 14:43:27,072 ERROR com.cloudera.cmon.firehose.Main: Error creating LevelDB timeseries store in directory /var/lib/cloudera-service-monitor/ts
java.io.IOException: Unable to create new directory at /var/lib/cloudera-service-monitor/ts/ts_entity_metadata
at com.cloudera.cmon.tstore.leveldb.LDBUtils.openVersionedDB(LDBUtils.java:239)
at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesMetadataStore.openMetadataDB(LDBTimeSeriesMetadataStore.java:149)
at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesMetadataStore.<init>(LDBTimeSeriesMetadataStore.java:139)
at com.cloudera.cmon.firehose.Main.main(Main.java:450)

Creating /var/lib/cloudera-service-monitor and /var/lib/cloudera-host-monitor directories manually solved my issue.

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

swapko — Wed, 07 Apr 2021 08:47:46 GMT

I am getting the same issue. However, I am not getting the option of cloudera management services to restart on UI< attached is the screenshot. Do we need to install and configure anything else apart from cloudera-scm-server and cloudera-scm-agent?

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

VidyaSargur — Wed, 07 Apr 2021 16:50:38 GMT

@swapko as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.

Re: CM5 Service Monitor Failed & Could Not Connect to Host Monitor Errors

vec — Mon, 08 Jul 2024 07:07:22 GMT

I run into same issue after deploying cdp 7.1.7. Is there workaround for this issue ?