Report Manager keeps going down with GC pauses

[#1]

Hello,

We have an issue in one of our environment, where the report manager keeps going down with GC pauses, this scenario is still evident after revising the heap twice for this service. Keen to understand what are the factors which contribute to this service's load which is leading to such a scenario. Also is increasing the heap is the only resort or GC algorithm it uses by default can be revisited? Please let me know your thoughts.

Thanks & Regards

Pravdeep

[#2]

A large fsimage that Reports Manager needs to index is the biggest cause for memory consumption. How large is your HDFS? Also if you don't plan to access reports, you could leave Reports Manager role stopped.

[#3]

Hello,

HDFS isn't very large at the moment. Utilisation stands at 25 TB out of total 420 TB.

Thanks & Regards

Pravdeep

[#4]

Hi All,

Turns out it was an RHEL bug on the node where report manager role is hosted and not related to the capacity, since as I stated above about HDFS utilisation. We had increased heap for this role number of items and everytime it would still jump resident memory threshold.

RHEL Bug “Random JVM Hangs on RHEL 6.6 (Kernels 3.14-3.17)” http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_os_ki.html

Thanks & Regards

Pravdeep

[#5]

We have the same issue of Reports manger going down frequently with out of memory error. The fsimage size is around 2.7Gb and Dfs used is 3.23Pb(86.27%). Server details below:

Linux 2.6.32-573.26.1.el6.x86_64

RHEL 6.7

Currently heap is increased to 6Gb but we are still experiencing issues while generating reports. Please advise.

Thanks,

Naigy

[#6]

Hello,

A good place to start is usually (4 x fsimage_size) + 3GB

Try about 15GB and see if that stabilizes Reports Manager.

Regards,

Ben