08-24-2018 04:25 AM
I'm curious what you all have set for Java Heap Size for Impala Catalog. At present i've had to increase to 128GB as 64GB limit was hit way to often causing OOM errors.
Total number of tables is 11,138 which are made up of 575,692 partitions
08-24-2018 06:32 AM
I checked additionaly the mem metrics, here are the details:
memory.mapped-bytes 5.14 GB Total bytes of memory mappings in this process (the virtual memory size). memory.rss 3.01 GB Resident set size (RSS) of this process, including TCMalloc, buffer pool and Jvm. memory.total-used 153.57 MB Total memory currently used by TCMalloc and buffer pool.
So practically it has 5.14GB in my case... It's quite a lot..But as you said, it has to probably fit the entire db into a memory, because this is the purpose of this service, right?