question Re: Should Impala release memory after use? in Archives of Support Questions (Read Only)

Should Impala release memory after use?

chriswalton007 — Fri, 16 Sep 2022 12:04:52 GMT

As per the title really. I'm seeing high memory usage on a few data nodes ,which is down to impala, however at the time there are no active impala queieres running on the nodes.

If I restart the impalad the memory will clear and remaing free until a query runs. I can monitor the query using the memory, but when it finishes the memory is not released back to the system.

Re: Should Impala release memory after use?

mathieu.d — Fri, 11 Aug 2017 13:15:02 GMT

As far as I understand how Impala works, that is the expected behaviour.

It is indeed intended for speeding up later queries that use the same sets of data.

Re: Should Impala release memory after use?

Tim Armstrong — Fri, 11 Aug 2017 14:55:57 GMT

It's normal for idle Impala daemons to hold onto 1-2GB of memory plus the JVM heap memory. Any more than that may indicate something wrong.

The memz debug page has diagnostics for this: http://impala-daemon:25000/memz?detailed=true . You can see there if a fragment of a query is holding onto memory.

There are two common ways this can happen:

* The query wasn't cancelled and closed. In this case it should show up on the /queries page of the coordinator

* The query was cancelled and closed, but a fragment continued running (this is a bug). In that case you should see a running fragment-execution thread in /threadz on the impala daemon web page.

Re: Should Impala release memory after use?

chriswalton007 — Fri, 11 Aug 2017 14:57:37 GMT

Thank you for the detailed answer