Member since
03-15-2022
3
Posts
0
Kudos Received
0
Solutions
03-07-2023
08:31 PM
I am trying to import a cluster template json file to form a cluster in CDP. I am unclear about the meaning of "base: true" in the service role definition blocks. { "refName": "impala-IMPALAD-BASE", "roleType": "IMPALAD", "base": true }, It is not documented on CDP website either.
... View more
Labels:
- Labels:
-
Cloudera Data Platform (CDP)
03-23-2022
05:26 AM
1 Kudo
Hi @PratCloudDev , "Spill to disk" happens when there is no enough memory available for a running query, Below is the example. Suppose you are running a query which is using 10gb ( per node peak memory) of memory and in case this query needs 12 gb of memory in this situation spill to disk happen on the configured scratch directories. you can see the directory by searching the "Impala Daemon Scratch Directories" property in the impala configurations. If you do not want to fail the query then you need to make sure the configured scratch directories/disk has enough space to store spilling information, this can potentially be large amounts of data. Check the query profile for "per node peak memory" it is the actual memory used for that query on each daemon, suppose if it is 15GB then set the MEM_LIMIT to 10gb or 12gb to see the spill to disk functionality. To understand why you are seeing the error[1] i need few details from your side. 1. Screenshot of impala admission control pool settings. 2. How much memory you are setting and seeing the below error[1]? 3. Which pool you are using to run the query? 4. If possible you can provide the query profile. Regards, Chethan YM [1]. Rejected query from pool root.centos: minimum memory reservation is greater than memory available to the query for buffer reservations.
... View more