question Re: impala java heap config in Archives of Support Questions (Read Only)

impala java heap config

ponypony — Fri, 16 Sep 2022 12:04:43 GMT

HI,

In impala config, i am setting mem limit to 24G, there is also a java heap size, i dont know if this is used by impala queries? and whats the relation with mem limit, and what should i set, if i have physical say 32G, how big can i set on java heap?

Thanks

Shannon

Re: impala java heap config

csguna — Fri, 11 Aug 2017 05:33:33 GMT

The best practice is to set 8 gb aside for the os and remaining for the dameons .

Are you refering to JAVA_TOOL_OPTIONS property .

I would start with 8G bump it when hit outofMemory during intial setup .

Re: impala java heap config

ponypony — Fri, 11 Aug 2017 12:00:39 GMT

Java Heap Size of Catalog Server in Bytes - i set to 4G

Java Heap Size of Impala Llama ApplicationMaster in Bytes - does this have anything to do with impala query?

Re: impala java heap config

Tim Armstrong — Fri, 11 Aug 2017 15:15:04 GMT

Llama is deprecated and shouldn't be in use - you should be safe to ignore that option.

Re: impala java heap config

stigahuang — Fri, 26 Apr 2019 08:36:50 GMT

Hi Tim,

Looks like there're no places to control the max JVM heap size of impalads in Cloudera Manager. (Only one for the catalogd) How can we set JAVA_TOOL_OPTIONS for impalads (coordinators)?

Re: impala java heap config

Tim Armstrong — Fri, 26 Apr 2019 15:58:03 GMT

We have "Java Heap Size of Impala Daemon in Bytes" in CM 6.1+ and 5.16+. Before that you had to use the "Impala Daemon Environment Advanced Configuration Snippet" safety valve. Here's an example from an internal cluster with dedicated coordinators set up.

Re: impala java heap config

Donal_RC — Fri, 21 Jun 2024 09:42:45 GMT

@stigahuang wrote:
Looks like there're no places to control the max JVM heap size of impalads in Cloudera Manager. (Only one for the catalogd) How can we set JAVA_TOOL_OPTIONS for impalads (coordinators)?

In CM 7.6.7, there is a configuration option called "impalad_embedded_jvm_heapsize". Not sure when it was added.

impalad_embedded_jvm_heapsize: Maximum size in bytes for the Java Process heap memory. Passed to Java -Xmx.