Service Monitor Heap size issue

[#1]

From last few days I am received alert from SERVICE_MONITOR_HEAP_SIZE in our CDP cluster (version 7.1.7). we are getting frequent alerts on the SERVICE_MONITOR_HEAP_SIZE.

How can I fix this issue? 

Monitored Entities are :

logs : 

[#2]

Hello @soumM 

Thank you for reaching out

Do you see GC paused in SMON logs? Can you try increasing the java heap size to 4GB

Regarding the connection timeout messages try performing a telnet from SMON host to the host on which you are observing in the logs (hostname and port)

# telnet <target-hostname> <port>

[#3]

Hi @upadhyayk04  ,

The memory has been upped to 6GB, and now there's a configuration override in place. Can you clarify what this override involves? 

 

[#4]

Hello,

Thank you for reaching out

The override is there so that if you have a role running on 2-3 nodes and you want to override a configuration on a particular host you can do that. If you want same for all nodes then you can set that up in Default group

[#5]

Hi @upadhyayk04, there is only one host on which the service is running, and the override is created for that host only while editing configuration service wide and default group is already set and host on which the service is running is the part of that group so ideally while changing the property it should have changed at host without creating override.

Can you please help?

[#6]

Hello @soumM 

Yes that is pretty strange, I don't how you have configured this

What version of CM CDP are you using?

Check CM UI >> Cloudera Management Services >> Configuration >> Roll Group and check what do you see there?