Should Impala release memory after use?

[#1]

As per the title really.  I'm seeing high memory usage on a few data nodes ,which is down to impala, however at the time there are no active impala queieres running on the nodes.

If I restart the impalad the memory will clear and remaing free until a query runs.  I can monitor the query using the memory, but when it finishes the memory is not released back to the system.

[#2]

As far as I understand how Impala works, that is the expected behaviour.

It is indeed intended for speeding up later queries that use the same sets of data.

[#3 · Accepted solution]

It's normal for idle Impala daemons to hold onto 1-2GB of memory plus the JVM heap memory. Any more than that may indicate something wrong.

The memz debug page has diagnostics for this: http://impala-daemon:25000/memz?detailed=true . You can see there if a fragment of a query is holding onto memory.

There are two common ways this can happen:

* The query wasn't cancelled and closed. In this case it should show up on the /queries page of the coordinator

* The query was cancelled and closed, but a fragment continued running (this is a bug). In that case you should see a running fragment-execution thread in /threadz on the impala daemon web page.

[#4]

Thank you for the detailed answer