question Re: Service Monitor Timeout in Archives of Support Questions (Read Only)

Service Monitor Timeout

sdbigdata — Thu, 13 Nov 2014 17:30:51 GMT

Since upgrading to 5.2 we occasionally receive:

"Request to the Service Monitor failed. This may cause slow page responses"

And the logs show the following:

com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 3687ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=3876ms

Happens pretty frequently.

Anyone else experiencing this / have any ideas? Never had this issue until 5.2 upgrade.

Re: Service Monitor Timeout

Grizzly — Thu, 13 Nov 2014 18:31:23 GMT

Increase heap for the Service monitor and restart the service would be the first thing to try.

Re: Service Monitor Timeout

sdbigdata — Fri, 14 Nov 2014 01:14:43 GMT

Thanks had tried that a few increments, but it didn't seem to help. Upped it up again, and now things do appear better.

CDH 5.1 we had it set to 1GB, never had this issue.

CDH 5.2 it's now set to 4GB, and so far it's looking OK. 1-3GB exhibited the issue.

4X the memory between versions seems high but hopefully that keeps it happy.

Thanks

Re: Service Monitor Timeout

Prince09 — Thu, 17 Nov 2016 19:34:58 GMT

What steps to take, if there is no change in the logs reported even after the heap size for service monitor is increased..?

Re: Service Monitor Timeout

bhavesh_gadoya8 — Tue, 06 Jun 2017 11:47:40 GMT

In my case, i am not even able to restart the service monitor service, as due to GC not detected. so unable to reflect changes to increase heap size of service monitor... Any other suggestion ?

Re: Service Monitor Timeout

shakti1 — Mon, 16 Oct 2017 23:01:05 GMT

worked for me, thanks!

Re: Service Monitor was failing

kvinod — Mon, 18 Feb 2019 06:25:15 GMT

Hello,

Service monitor was failing with below error messages and i can see below warninings in service monitor logs,

ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper,

WARN com.cloudera.enterprise.debug.JvmPauseMonitor
Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 21465ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=21606ms

Can some one please respond back.

Thanks,

Vinod

Re: Service Monitor was failing

kvinod — Thu, 28 Mar 2019 10:53:54 GMT

Hello Can any one please respond on this issue,

i am seeing frequent service monitor failures and getting it paused for some time and then it comes online.

Thanks in advance....!

Vinod

Re: Service Monitor was failing

gzigldrum — Thu, 28 Mar 2019 15:46:31 GMT

Frequent pauses in the JVM by the Garbage Collector indicate the heap memory settings are too low.

Please increase the CM -> Cloudera Management Service -> Configuration -> Scope: Service Monitor -> Category: Resource Management -> Java Heap Size of Service Monitor in Bytes and Java Heap Size of Service Monitor in Bytes configuration property values accordingly.

The Service Monitor Requirements documentation chapter has guidance for the values to choose.