question Re: SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad in Support Questions

SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad

Raj77 — Fri, 11 Dec 2020 14:46:37 GMT

SERVICE_MONITOR_AGGREGATION_RUN_DURATION The health test result for SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad: The last metrics aggregation run duration is 33.9 second(s). Critical threshold: 30 second(s).

Service Monitor File /var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR.log.out 8:35:51.569 PM WARN Groups Potential performance problem: getGroups(user=hue) took 8024 milliseconds. 8:36:21.270 PM WARN JvmPauseMonitor Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 18430ms: no GCs detected. 8:36:21.558 PM WARN EnterpriseService com.cloudera.cmf.PollingScmProxy: run duration exceeded desired period. Duration: 19230 ms. Desired period: 1000 ms. 8:36:22.253 PM INFO AggregatingTimeSeriesStore Run took PT33.916S which is over the slow run threshold of PT30S. 15690 metrics written for 28 entities. PT27.952S write time over 2 writes. Longest writes: PT27.876S,PT0.076S. 8:37:02.720 PM INFO LDBPartitionManager Updating partition=LDBPartitionMetadataWrapper{tableName=stream, partitionName=stream_2020-12-10T17:48:01.580Z, startTime=2020-12-10T17:48:01.580Z, endTime=null, version=2, state=OPEN}. Setting endTime=2020-12-10T18:38:02.709Z

Metrics Aggregation Run Duration Thresholds critical:30000.0, warning:10000.0 present we are using default values in my cluster. any changes required for this values. else any other issue is there in my cluster.

Re: SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad

GangWar — Fri, 11 Dec 2020 16:20:07 GMT

@Raj77 Your Service Monitor is running out of Java Heap hence the issue:

JvmPauseMonitor	Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 18430ms: no GCs detected.

So please increase the Java heap size for SMON to resolve the issue. Refer the Requirements here for Heap size: https://docs.cloudera.com/cloudera-manager/7.2.1/managing-clusters/topics/cm-configuring-memory-allocations.html

Re: SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad

Raj77 — Sat, 12 Dec 2020 04:41:14 GMT

My cluster Java Heap memory values kindly please check this values.

Raj@77 Your Service Monitor is running out of Java Heap hence the issue: where can i found exact location of service monitor using java heap memory value. is there any specific requirement for service monitor Java Heap value.

Is there any issue.

/etc/default/cloudera-scm-server Java Options. # # Default value sets Java maximum heap size to 2GB, and Java maximum permanent # generation size to 256MB. # export CMF_JAVA_OPTS="-Xmx2G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"

Heap size: https://docs.cloudera.com/cloudera-manager/7.2.1/managing-clusters/topics/cm-configuring-memory-allo... Link is not working

Re: SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad

GangWar — Mon, 14 Dec 2020 09:37:43 GMT

@Raj77 You have to go CM > Cloudera Management Services > Configuration > Then Form category filter select Service Monitor

Then you can just search for Heap size and you will get SMON heap size there.

Pasting The Link again: https://docs.cloudera.com/cloudera-manager/7.2.1/managing-clusters/topics/cm-configuring-memory-allocations.html

Re: SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad

chuvalski — Tue, 17 Jun 2025 17:57:09 GMT

Hi,

Here it's the link
https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/managing-clusters/topics/cm-configuring-memory-allocations.html