IMPALA - MEMORY LIMIT EXEDED

[#1]

Hi,

I’m experiencing an issue when multiple queries are executed simultaneously. When I run the query by itself, it succeeds. However, when multiple queries run concurrently, I encounter problems.

Does anyone know how I should configure Impala to avoid this issue? I’ve attached the profile of the failed query.

My mem_limit is set to 60GB for 5 workers and 90GB for the other 8.

Thanks,

[#2]

@Edu7478 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Impala experts @ChethanYM @Chella @mszurap  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

Hello @Edu7478 

This seems to be a common issue faced during high concurrency when there is no control on how much memory a query can utilise.

You can check below :

- all tables has updated stats

- configure / fine tume Impala Admission Control

[#4]

Hello @Saurabhatiyal,

Is there a guide or some tips we can follow to tune Impala Admision Control?

Thanks in advance,

[#5 · Accepted solution]

You can go through below 2 documentation to understand more on Impala Admission Control and how to configure it as per workload :
1. Parameters to set up for admission control

https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/impala-manage/topics/impala-admission.html

2. Sample Scenario

https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/impala-manage/topics/impala-admission-example...

[#6]

@Edu7478 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.