I am trying to import a cluster template json file to form a cluster in CDP. I am unclear about the meaning of "base: true" in the service role definition blocks.  { "refName": "impala-IMPALAD-BASE", "roleType": "IMPALAD", "base": true },   It is not documented on CDP website either.  ... View more

I am trying to trigger spilling to disk on impala (Impala Shell v3.2.0-cdh6.3.4 )  The cloudera documentation suggest the steps to do it here: https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/impala_scalability.html Under the section: "Testing performance implications of spilling to disk:"   I am extremely confused by the two seemingly contradictory statements there: "Set the MEM_LIMIT query option to a value that is smaller than the peak memory usage reported in the profile output. Do not specify a memory limit lower than reported in the profile output. " So what should be the value of MEM_LIMIT ? When I do set it to be lower than the peak memory usage, i get the following error : "Rejected query from pool root.centos: minimum memory reservation is greater than memory available to the query for buffer reservations. Memory reservation needed given the current...." When I set the MEM_LIMIT to be higher, then I don't see any "SpilledPartitions" or "SpilledRuns" counters, that is, not triggering any spilled to disk.  Since the documentation is unclear, I want to know how I can trigger spill to disk functionality without the queries failing ?   ... View more