Support Questions
Find answers, ask questions, and share your expertise

Need root cause of GC Allocation Failure in hadoop gc log

New Contributor

Hi,

Is anybody familiar with the below error thrown at hadoop gc.log file.

It would be great help if any of you can provide some direction to find out the root cause of this error.

cluster version : HDP-3.1.5.0

hadoop version : 3.1.1.3.1

log details from gc.log file : 

2022-04-19T07:08:01.716+0000: 44976.365: [GC (CMS Initial Mark) [1 CMS-initial-mark: 10482589K(13303808K)] 11540107K(15014336K), 0.0814416 secs] [Times: user=0.63 sys=0.01, real=0.09 secs]
2022-04-19T07:08:01.798+0000: 44976.447: [CMS-concurrent-mark-start]
2022-04-19T07:08:05.842+0000: 44980.491: [GC (Allocation Failure) 2022-04-19T07:08:05.843+0000: 44980.492: [ParNew: 1565948K->41595K(1710528K), 0.0406024 secs] 12048537K->10528136K(15014336K), 0.0413880 secs] [Times: user=0.25 sys=0.04, real=0.04 secs]
2022-04-19T07:08:11.691+0000: 44986.340: [CMS-concurrent-mark: 9.816/9.893 secs] [Times: user=21.19 sys=2.28, real=9.89 secs]
2022-04-19T07:08:11.691+0000: 44986.340: [CMS-concurrent-preclean-start]
2022-04-19T07:08:11.742+0000: 44986.391: [CMS-concurrent-preclean: 0.049/0.051 secs] [Times: user=0.05 sys=0.01, real=0.05 secs]
2022-04-19T07:08:11.742+0000: 44986.391: [CMS-concurrent-abortable-preclean-start]
CMS: abort preclean due to time 2022-04-19T07:08:16.848+0000: 44991.497: [CMS-concurrent-abortable-preclean: 5.086/5.106 secs] [Times: user=6.53 sys=0.31, real=5.11 secs]
2022-04-19T07:08:16.852+0000: 44991.501: [GC (CMS Final Remark) [YG occupancy: 1422222 K (1710528 K)]2022-04-19T07:08:16.852+0000: 44991.501: [Rescan (parallel) , 0.1072305 secs]2022-04-19T07:08:16.959+0000: 44991.608: [weak refs processing, 0.0000988 secs]2022-04-19T07:08:16.960+0000: 44991.609: [class unloading, 0.0443750 secs]2022-04-19T07:08:17.004+0000: 44991.653: [scrub symbol table, 0.0149902 secs]2022-04-19T07:08:17.019+0000: 44991.668: [scrub string table, 0.0015677 secs][1 CMS-remark: 10486540K(13303808K)] 11908763K(15014336K), 0.1685675 secs] [Times: user=0.88 sys=0.04, real=0.17 secs]
2022-04-19T07:08:17.021+0000: 44991.670: [CMS-concurrent-sweep-start]
2022-04-19T07:08:18.223+0000: 44992.872: [GC (Allocation Failure) 2022-04-19T07:08:18.223+0000: 44992.872: [ParNew: 1562107K->40307K(1710528K), 0.0386524 secs] 12048585K->10530723K(15014336K), 0.0393966 secs] [Times: user=0.26 sys=0.01, real=0.04 secs]
2022-04-19T07:08:20.732+0000: 44995.381: [CMS-concurrent-sweep: 3.658/3.711 secs] [Times: user=4.87 sys=0.22, real=3.71 secs]
2022-04-19T07:08:20.732+0000: 44995.381: [CMS-concurrent-reset-start]
2022-04-19T07:08:20.764+0000: 44995.413: [CMS-concurrent-reset: 0.031/0.031 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2022-04-19T07:08:22.768+0000: 44997.417: [GC (CMS Initial Mark) [1 CMS-initial-mark: 10484352K(13303808K)] 11039988K(15014336K), 0.0472202 secs] [Times: user=0.36 sys=0.01, real=0.04 secs]
2022-04-19T07:08:22.816+0000: 44997.465: [CMS-concurrent-mark-start]
2022-04-19T07:08:31.292+0000: 45005.941: [GC (Allocation Failure) 2022-04-19T07:08:31.292+0000: 45005.941: [ParNew: 1560819K->48000K(1710528K), 0.0364257 secs] 12045172K->10536353K(15014336K), 0.0372010 secs] [Times: user=0.24 sys=0.00, real=0.04 secs]
2022-04-19T07:08:33.420+0000: 45008.069: [CMS-concurrent-mark: 10.499/10.604 secs] [Times: user=21.98 sys=2.85, real=10.61 secs]
2022-04-19T07:08:33.420+0000: 45008.069: [CMS-concurrent-preclean-start]
2022-04-19T07:08:33.472+0000: 45008.121: [CMS-concurrent-preclean: 0.050/0.052 secs] [Times: user=0.06 sys=0.00, real=0.05 secs]
2022-04-19T07:08:33.472+0000: 45008.121: [CMS-concurrent-abortable-preclean-start]

 

Regards

Abhishek

0 REPLIES 0
; ;