impala catalog server JVM

[#1]

Hi.

I have a problem with outOfMemory error on impala catalog server.

Data in tables about 30TB and about 175k partitiions.

I set Java Heap Size of Catalog Server in Bytes=30gb, but still have out_of_memory error in start.

How can i calculate how much memory needs for catalog server? And is it have any limit to set ?

[#2 · Accepted solution]

Slide 17 here has some rules of thumb - https://blog.cloudera.com/latest-impala-cookbook/

Can you mention what version you're running and whether you have any other non-standard configs set, e.g. load_catalog_in_background. We made some improvements in this area and have added some options in more recent versions.