question Re: Impala Catalogue Server OOM in Support Questions

Impala Catalogue Server OOM

Amn_468 — Fri, 16 Sep 2022 14:43:07 GMT

Hello,

We are observing our Impala Catalogue Server’s process frequently gets exited / killed (This role encountered 1 unexpected exit(s) in the previous 5 minute(s).This included 1 exit(s) due to OutOfMemory errors. Critical threshold: any.)

I was going through this article (https://community.cloudera.com/t5/Support-Questions/Cloudera-6-2-1-Impala-GC-Overhead-limit-Exceeded/td-p/293225) it seems to be Heap Memory related, would like to know if there is any way / calculations to find how much Heap should be allocated to avoid these issues.

CM / CDH 5.16.2

Java Heap Size of Catalog Server in Bytes = 15Gb

Appreciate any guidance in this regard.

Re: Impala Catalogue Server OOM

robbiez — Tue, 24 Aug 2021 00:48:23 GMT

There is a calculation in The Impala Cookbook to estimate the heap memory usage for metadata:

• num of tables * 5KB + num of partitions * 2KB + num of files * 750B + num of file blocks * 300B + sum(incremental col stats per table)

• Incremental stats

For each table, num columns * num partitions * 400B

Usually, the insufficient catalog heap memory is caused by a large number of small files or/and partitions. For example, a single 512MB file needs (750B + 4 * 3 * 300B) = 4350B. But if we split this file to 128 4MB files, these files will use (128 * 750B + 128 * 3 * 300B) = 211200B, nearly 50 times!

Re: Impala Catalogue Server OOM

balajip — Tue, 09 Sep 2025 13:50:37 GMT

@Amn_468

Please refer the below Knowledge base article for heap memory calculation.

https://lighthouse.cloudera.com/s/article/Impaladembeddedjvmheapsize-has-a-default-value-of-32-GB-after-upgrading-to-CDH-516https://my.cloudera.com/knowledge/Impaladembeddedjvmheapsize-has-a-default-value-of-32-GB-after?id=278279

Re: Impala Catalogue Server OOM

VidyaSargur — Thu, 26 Aug 2021 04:34:37 GMT

@Amn_468 Has any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.