Max Query Mem Default doesn't working

solardelune — Wed, 01 Jul 2026 13:15:01 GMT

I have set my impala.admission-control.min-query-mem-limit.root.default to 2GiB and my impala.admission-control.max-query-mem-limit.root.default to 57GiB, which aligned with the memory allocation for each Impala Executor node that I've set on the resource templates.

My problem is, when I'm running a heavy query (I tried to set it so the limit is 10000000), the limit memory is set to 2GiB and makes the query OOM, which is strange because the max-query-mem-limit is set to 57GiB, so the limit should be > 2GiB and <= 57GiB if the query is heavy, right? Yet it's just set to the minimum mem limit.

I know that the limit is set to clamp(min memory limit per query, memory estimated, max memory limit per query), but the estimated memory is 4GB and what makes it stranger is that when I try to define SET MEM_LIMIT in the query to another number like 4G or 8G, the estimated memory changes; which means there's some calculation for the estimated memory based on the memory limit.

And that's why my question is: why can't my daemon use the max-memory-limit that i've set and just set it based on the min-memory-limit?

Re: Max Query Mem Default doesn't working

RAGHUY — Thu, 02 Jul 2026 04:17:02 GMT

@solardelune

This is a great question. The behavior you are observing is actually working exactly as designed by Impala's Admission Controller. It comes down to how the admission controller "clamps" user limits and how the query planner interacts with the MEM_LIMIT parameter.

Here is exactly why your query is clamping to 2GiB, why the estimates change, and how to fix it
1. Why your limit was set to 2GiB (The Clamping Behavior) When you configure min-query-mem-limit (2GiB) and max-query-mem-limit (57GiB), you establish a strict memory floor and ceiling for queries running in that specific resource pool

You mentioned that you tried setting the limit to 10000000. Because MEM_LIMIT takes values in bytes if no suffix is provided, 10000000 evaluates to just 10 MB

By default, Impala has a setting called Clamp MEM_LIMIT Query Option enabled
. If a user sets a MEM_LIMIT that falls outside of your configured min/max range, the Admission Controller forcefully "clamps" the limit back into the allowed boundaries
. Because your requested 10 MB fell far below the 2GiB minimum, the admission controller clamped it up to your 2GiB floor
. Since this was a heavy query, 2GiB was not enough memory to complete the operation, resulting in an Out-Of-Memory (OOM) error.

2. Why the memory estimate changes when you SET MEM_LIMIT It is completely normal that your estimated memory changes when you manually execute SET MEM_LIMIT=4G or 8G

When you set an explicit MEM_LIMIT, Impala's query planner takes this hard boundary into account when constructing the execution strategy. Knowing the memory ceiling changes how the planner assigns internal resources—such as when to activate the "spill-to-disk" mechanism or how to size internal memory buffers
. Because the underlying query plan adapts to accommodate your new limit, the resulting memory estimate changes as well. In fact, the Impala error logs explicitly warn that "changing the mem_limit may also change the plan"

3. How to make the daemon use the higher memory limits To stop your queries from OOMing and allow them to utilize up to your 57GiB maximum, you have two options:
Option A (Manual Override): Explicitly set the MEM_LIMIT for your heavy query to a value between 2GiB and 57GiB using the correct size suffixes (e.g., SET MEM_LIMIT=30G;)
. Because 30 GB is comfortably within your boundaries, the Admission Controller will not clamp it, and your query will be granted the 30 GB needed to execute safely

Option B (Automatic Estimation via Statistics): If you do not set a MEM_LIMIT, the Admission Controller relies entirely on the planner's memory estimate to decide how much memory to set aside
. If your heavy query is only being assigned a 4GB estimate automatically but requires much more, it means your table statistics are missing, stale, or corrupt. You must run the COMPUTE STATS statement on the tables involved in the query
. Accurate statistics allow the planner to correctly calculate a much higher memory estimate, which the Admission Controller will then automatically grant within your 2GiB to 57GiB window

question Max Query Mem Default doesn't working in Support Questions

Max Query Mem Default doesn't working

Re: Max Query Mem Default doesn't working