Support Questions
Find answers, ask questions, and share your expertise

Report Manager keeps going down with GC pauses

Report Manager keeps going down with GC pauses

Explorer

Hello,

 

We have an issue in one of our environment, where the report manager keeps going down with GC pauses, this scenario is still evident after revising the heap twice for this service. Keen to understand what are the factors which contribute to this service's load which is leading to such a scenario. Also is increasing the heap is the only resort or GC algorithm it uses by default can be revisited? Please let me know your thoughts.

 

 

Thanks & Regards

Pravdeep

5 REPLIES 5
Highlighted

Re: Report Manager keeps going down with GC pauses

Cloudera Employee

A large fsimage that Reports Manager needs to index is the biggest cause for memory consumption. How large is your HDFS? Also if you don't plan to access reports, you could leave Reports Manager role stopped.

Highlighted

Re: Report Manager keeps going down with GC pauses

Explorer

Hello,

 

HDFS isn't very large at the moment. Utilisation stands at 25 TB out of total 420 TB.

 

Thanks & Regards

Pravdeep

Highlighted

Re: Report Manager keeps going down with GC pauses

Explorer

Hi All,

 

Turns out it was an RHEL bug on the node where report manager role is hosted and not related to the capacity, since as I stated above about HDFS utilisation. We had increased heap for this role number of items and everytime it would still jump resident memory threshold.

 

RHEL Bug “Random JVM Hangs on RHEL 6.6 (Kernels 3.14-3.17)” http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_os_ki.html

 

Thanks & Regards

Pravdeep

Highlighted

Re: Report Manager keeps going down with GC pauses

New Contributor

We have the same issue of Reports manger going down frequently with out of memory error. The fsimage size is around 2.7Gb and Dfs used is 3.23Pb(86.27%). Server details below:

 

Linux 2.6.32-573.26.1.el6.x86_64

RHEL 6.7

 

Currently heap is increased to 6Gb but we are still experiencing issues while generating reports. Please advise.

 

Thanks,

Naigy

Highlighted

Re: Report Manager keeps going down with GC pauses

Super Guru

Hello,

 

A good place to start is usually (4 x fsimage_size) + 3GB

Try about 15GB and see if that stabilizes Reports Manager.

 

Regards,

 

Ben