Support Questions

Find answers, ask questions, and share your expertise

Impala Catalog JVM Size

Rising Star

Hi All,

I'm curious what you all have set for Java Heap Size for Impala Catalog.  At present i've had to increase to 128GB as 64GB limit was hit way to often causing OOM errors.

Total number of tables is 11,138 which are made up of 575,692 partitions



Rising Star

Another note is that total size of the MySQL HiveMetastore database is 102GB, so maybe need to be simialr to that.

Master Collaborator
That seemed to me too much, but I checked that for ~15k partitions I have 2.5GB, so ~150k per partition.

Rising Star

Seems about right then based upon your values.  Good to know i've not got some weird memory leak going on or something


Master Collaborator

I checked additionaly the mem metrics, here are the details:

memory.mapped-bytes 5.14 GB Total bytes of memory mappings in this process (the virtual memory size).
memory.rss 3.01 GB Resident set size (RSS) of this process, including TCMalloc, buffer pool and Jvm. 153.57 MB Total memory currently used by TCMalloc and buffer pool.

So practically it has 5.14GB in my case... It's quite a lot..But as you said, it has to probably fit the entire db into a memory, because this is the purpose of this service, right?


Rising Star

Yes that's my understanding of what the catalog is used for.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.