Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Impala - Memory limit exceeded

Labels:
avatar
author-rank Johnny_Bach
Contributor

Created on ‎09-27-2018 10:58 PM - edited ‎09-16-2022 06:45 AM

i'm receving the below error message on memory limit although there is enough default memory limit set on the resource pool and explan plan only shows 288 MB out of 18 nodes cluster which leads to 5184 MB of total memory consumption

 

+-----------------------------------------------------------+
| Explain String |
+-----------------------------------------------------------+
| Estimated Per-Host Requirements: Memory=288.00MB VCores=1 |
| |
| 01:EXCHANGE [UNPARTITIONED] |
| | limit: 1 |
| | |
| 00:SCAN HDFS [fenet5.hmig_os_changes_details_malicious] |
| partitions=1/25 files=3118 size=110.01GB |
| predicates: job_id = 55451 |
| limit: 1 |
+-----------------------------------------------------------+

 

WARNINGS:
Memory limit exceeded
HdfsParquetScanner::ReadDataPage() failed to allocate 269074889 bytes for dictionary.

 

Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 257.23 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=20.00 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=20.00 GB
HDFS_SCAN_NODE (id=0): Consumption=20.00 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.63 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=20.00 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=20.00 GB
HDFS_SCAN_NODE (id=0): Consumption=20.00 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.27 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=20.00 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=20.00 GB
HDFS_SCAN_NODE (id=0): Consumption=20.00 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.39 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=20.00 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=20.00 GB
HDFS_SCAN_NODE (id=0): Consumption=20.00 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 16.09 KB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.74 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.74 GB
HDFS_SCAN_NODE (id=0): Consumption=19.74 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 15.20 KB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.64 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.64 GB
HDFS_SCAN_NODE (id=0): Consumption=19.64 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 14.61 KB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.64 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.64 GB
HDFS_SCAN_NODE (id=0): Consumption=19.64 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 257.11 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.47 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.47 GB
HDFS_SCAN_NODE (id=0): Consumption=19.47 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.51 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.24 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.24 GB
HDFS_SCAN_NODE (id=0): Consumption=19.24 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.32 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.24 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.24 GB
HDFS_SCAN_NODE (id=0): Consumption=19.24 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.73 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.49 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.49 GB
HDFS_SCAN_NODE (id=0): Consumption=19.49 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.29 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.49 GB
Fragment 294eb435fbf8fc63:f529602818758c8b: Consumption=19.49 GB
HDFS_SCAN_NODE (id=0): Consumption=19.49 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 256.61 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: memory limit exceeded. Limit=20.00 GB Consumption=20.00 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=20.00 GB
HDFS_SCAN_NODE (id=0): Consumption=20.00 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 256.05 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=19.69 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=19.69 GB
HDFS_SCAN_NODE (id=0): Consumption=19.69 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.35 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=17.97 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=17.97 GB
HDFS_SCAN_NODE (id=0): Consumption=17.72 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 1.02 KB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=17.63 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=17.63 GB
HDFS_SCAN_NODE (id=0): Consumption=17.63 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 1.01 KB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=16.94 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=16.94 GB
HDFS_SCAN_NODE (id=0): Consumption=16.68 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 88.00 KB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=16.61 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=16.61 GB
HDFS_SCAN_NODE (id=0): Consumption=16.36 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
HDFS_SCAN_NODE (id=0) could not allocate 255.23 MB without exceeding limit.
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=16.30 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=16.30 GB
HDFS_SCAN_NODE (id=0): Consumption=16.30 GB
DataStreamSender: Consumption=1.45 KB
Block Manager: Limit=16.00 GB Consumption=0
Memory Limit Exceeded
Query(294eb435fbf8fc63:f529602818758c80) Limit: Limit=20.00 GB Consumption=8.02 GB
Fragment 294eb435fbf8fc63:f529602818758c8e: Consumption=8.02 GB
HDFS_SCAN_NODE (id=0): Consumption=8.02 GB
Block Manager: Limit=16.00 GB Consumption=0

35,956 Views
0 Kudos
2 ACCEPTED SOLUTIONS

avatar
author-rank EricL author-rank
Super Guru

Created ‎10-09-2018 11:05 PM

What's the MEM_LIMIT set for this query? Have you tried to increase the limit for this session to see if it helps?

SET MEM_LIMIT=100g;

for example.

View solution in original post

35,923 Views
0 Kudos
1

avatar
author-rank Tim Armstrong author-rank
Guru

Created ‎11-28-2018 10:31 AM

@alexmc6 I think there's (understandably) some misunderstanding of what the different mechanisms there do. 


Memory estimates only play a role if you set "Max Memory" and leave "Default Query Memory Limit" unset or set to 0. I always recommend against that mode for exactly the reason you mentioned.

View solution in original post

35,402 Views
0 Kudos
10 REPLIES 10

avatar
author-rank newbieone
Explorer

Created ‎08-18-2022 12:52 AM

Can we set 100g? Or maybe 1000g?

8,765 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera DataFlow 2.9 now supports building GenAI pipelines ...
What's New @ Cloudera
Accelerate Data Scientist Productivity with Cloudera Copilot...
What's New @ Cloudera
[RELEASED] Cloudera Streaming Analytics - Kubernetes Operato...
Community Announcements
October 2024 Community Highlights
What's New @ Cloudera
Accelerate Your AI with the Cloudera AI Inference Service wi...
View More Announcements