Skip to main content
Edu7478
Edu7478
Silver ⭐️
July 23, 2024
Solved

IMPALA - MEMORY LIMIT EXEDED

Hi,

I’m experiencing an issue when multiple queries are executed simultaneously. When I run the query by itself, it succeeds. However, when multiple queries run concurrently, I encounter problems.

Does anyone know how I should configure Impala to avoid this issue? I’ve attached the profile of the failed query.

My mem_limit is set to 60GB for 5 workers and 90GB for the other 8.

Thanks,

    Saurabhatiyal
    SaurabhatiyalAnswer
    Gold ⭐️
    July 24, 2024
    You can go through below 2 documentation to understand more on Impala Admission Control and how to configure it as per workload :
    1.     Parameters to set up for admission control

    2. Sample Scenario
    View in thread

    5 replies

    DianaTorres
    Community Manager
    DianaTorres
    Community Manager
    July 23, 2024

    @Edu7478 Welcome to the Cloudera Community!

    To help you get the best possible solution, I have tagged our Impala experts @ChethanYM @Chella @mszurap  who may be able to assist you further.

    Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

    Translate
    Regards, Diana Torres, Senior Community Moderator
    Saurabhatiyal
    Gold ⭐️
    Saurabhatiyal
    Gold ⭐️
    July 23, 2024

    Hello @Edu7478 

    This seems to be a common issue faced during high concurrency when there is no control on how much memory a query can utilise.

    You can check below :

    - all tables has updated stats

    - configure / fine tume Impala Admission Control

    Translate
    Edu7478
    Edu7478Author
    Silver ⭐️
    July 24, 2024

    Hello @Saurabhatiyal,

    Is there a guide or some tips we can follow to tune Impala Admision Control?

    Thanks in advance,

    Translate
    Saurabhatiyal
    Gold ⭐️
    SaurabhatiyalAnswer
    Gold ⭐️
    July 24, 2024
    You can go through below 2 documentation to understand more on Impala Admission Control and how to configure it as per workload :
    1.     Parameters to set up for admission control

    2. Sample Scenario
    Translate
      DianaTorres
      Community Manager
      DianaTorres
      Community Manager
      July 29, 2024

      @Edu7478 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.

      Translate
      Regards, Diana Torres, Senior Community Moderator
      Terms & ConditionsAccessibility statement