Reply
Highlighted
Explorer
Posts: 15
Registered: ‎03-03-2017

Impala memory configuration

Dear All,

 

I am using Cloudera 14 version. Cluster size is of 30 tb 8 node cluster

 

Issue is "After couple of query executions in impala hue/console getting conole memory exceeded by 131 mb error" or message like "Query expired or Rerun the Query if required"

 

Configuration of Impala is done by AWS load balancer .

 

Please suggest the memory recommened approach or any bechmark that can be configured in Impala . 

 

 

Thanks and Regards,

Naveen

Cloudera Employee
Posts: 375
Registered: ‎03-23-2015

Re: Impala memory configuration

Couple of things to note:

1. How much memory you allocate to impala daemon? Can you give more?

2. What's the setting for impala query and session timeout? You might want to reduce the timeout so that resources can be released sooner (Hue keeps query handler open until query/session closed)

3. Have you enabled admission control?
https://www.cloudera.com/documentation/enterprise/latest/topics/admin_impala_admission_control.html

You can control how many queries to run at a time depending on resource consumption.

Explorer
Posts: 15
Registered: ‎03-03-2017

Re: Impala memory configuration

Yes Admission control is enabled now

Cloudera Employee
Posts: 375
Registered: ‎03-23-2015

Re: Impala memory configuration

"Query expired or Rerun the Query if required" means the query or session has expired at Impala level, I would suggest you review those settings, maybe you need to increase those values.

Regarding OOM error, can you also check if you have table stats for those tables the query run against? If the table stats are missing or incorrect, impala can produce wrong estimates and admits query through, but in reality, it could need to use far more memory to run, hence causes the OOM error.

Announcements