Impala Catalogue Server OOM

[#1]

Hello,

We are observing our Impala Catalogue Server’s process frequently gets exited / killed (This role encountered 1 unexpected exit(s) in the previous 5 minute(s).This included 1 exit(s) due to OutOfMemory errors. Critical threshold: any.)

I was going through this article (https://community.cloudera.com/t5/Support-Questions/Cloudera-6-2-1-Impala-GC-Overhead-limit-Exceeded...) it seems to be Heap Memory related, would like to know if there is any way / calculations to find how much Heap should be allocated to avoid these issues.

CM / CDH 5.16.2

Java Heap Size of Catalog Server in Bytes = 15Gb

Appreciate any guidance in this regard.

[#2 · Accepted solution]

There is a calculation in The Impala Cookbook to estimate the heap memory usage for metadata:

• num of tables * 5KB + num of partitions * 2KB + num of files * 750B + num of file blocks * 300B + sum(incremental col stats per table)

• Incremental stats

     For each table, num columns * num partitions * 400B

Usually, the insufficient catalog heap memory is caused by a large number of small files or/and partitions. For example, a single 512MB file needs (750B + 4 * 3 * 300B) = 4350B. But if we split this file to 128 4MB files, these files will use (128 * 750B + 128 * 3 * 300B) = 211200B, nearly 50 times!

[#3]

@Amn_468 

Please refer the below Knowledge base article for heap memory calculation. 

https://my.cloudera.com/knowledge/Impaladembeddedjvmheapsize-has-a-default-value-of-32-GB-after?id=2...

[#4]

@Amn_468 Has any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.