Cloudera manager 5.5.1 JVM hangs

RK — Fri, 16 Sep 2022 10:12:13 GMT

Around 10 days back we upgraded our Cloudera manager from Cloudera manager 4.8.5 to Cloudera Express 5.5.1. This was working fine until today and in last few hours we seeing cloudera manager hanging till it is restarted. Our cluster is in CDH4. Also after upgrade of cloudera manager, we haven't restarted the cluster yet, this might not matter.

Has anybody faced similar issue?

Thanks

Here are few lines from CM server log ...

2016-04-04 21:53:37,516 INFO JvmPauseMonitor:com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 5312ms: GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=5401ms

2016-04-04 21:53:44,972 INFO JvmPauseMonitor:com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1102ms: no GCs detected.

2016-04-04 21:53:55,157 INFO JvmPauseMonitor:com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 6389ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=6880ms

2016-04-04 21:54:12,226 INFO JvmPauseMonitor:com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1111ms: no GCs detected.

In GC logs

CMS: abort preclean due to time 2016-04-04T21:43:21.873-0700: 733.159: [CMS-concurrent-abortable-preclean: 2.699/5.030 secs] [Times: user=4.27 sys=0.27, real=5.03 secs]

2016-04-04T21:43:21.874-0700: 733.160: [GC[YG occupancy: 267551 K (629248 K)]2016-04-04T21:43:21.874-0700: 733.160: [Rescan (parallel) , 0.0272730 secs]2016-04-04T21:43:21.902-0700: 733.187: [weak refs processing, 0.0000870 secs]2016-04-04T21:43:21.902-0700: 733.187: [scrub string table, 0.0038900 secs] [1 CMS-remark: 1233151K(1443052K)] 1500703K(2072300K), 0.0313960 secs] [Times: user=0.59 sys=0.00, real=0.04 secs]

2016-04-04T21:46:02.021-0700: 893.307: [GC2016-04-04T21:46:02.021-0700: 893.307: [ParNew: 627919K->68971K(629248K), 0.0194360 secs] 9202459K->8712645K(12826296K), 0.0195760 secs] [Times: user=0.39 sys=0.01, real=0.02 secs]

CMS: abort preclean due to time 2016-04-04T21:46:02.780-0700: 894.066: [CMS-concurrent-abortable-preclean: 3.586/5.010 secs] [Times: user=4.94 sys=0.09, real=5.01 secs]

2016-04-04T21:55:39.345-0700: 1470.631: [Full GC2016-04-04T21:55:39.345-0700: 1470.631: [CMS2016-04-04T21:55:40.705-0700: 1471.991: [CMS-concurrent-mark: 1.359/1.362 secs] [Times: user=1.37 sys=0.00, real=1.36 secs]

(concurrent mode failure): 17825660K->17825627K(17825792K), 7.2681860 secs] 20656828K->20656272K(20656960K), [CMS Perm : 131326K->131326K(221484K)], 7.2683460 secs] [Times: user=7.28 sys=0.00, real=7.27 secs]

Re: Cloudera manager 5.5.1 JVM hangs

RK — Thu, 23 Jun 2016 01:18:50 GMT

This was resolved by increasing heap size temporarily for cloudera manager.

question Cloudera manager 5.5.1 JVM hangs in Archives of Support Questions (Read Only)

Cloudera manager 5.5.1 JVM hangs

Re: Cloudera manager 5.5.1 JVM hangs