Member since
08-07-2017
14
Posts
1
Kudos Received
0
Solutions
05-11-2018
02:51 PM
Hi @Tim Armstrong Hope you are doing well, It will be nice if we have a metirc for the memory part of the daemons_memory_limit used by impala daemon in a given time. So when i get a query failing on memory, i can investigate the memory usage thatwill help me to understand when to increase the limit, secondly, i can learn trend and usage over time and i can plan my increase. Currently i see only the resident memory per node but this memory isn't used by the queries, so it's a diffcult task for me to investigate the impala behaviour once a query failed on memory. Yes i have a metric of the total memory used by node, but i have different roles in the node, so it hard to track this issue.
... View more
08-14-2017
02:02 PM
Hi Tim, Thanks for the information. A minor observation: I found that "tcmalloc_physical_bytes_reserved_across_impalads" and "mem_tracker_process_limit_across_impalads" are for v5.3.x (ref: https://www.cloudera.com/documentation/enterprise/5-3-x/topics/cm_metrics_impala.html) and "tcmalloc_physical_bytes_reserved" and "mem_tracker_process_limit" are for v5.7.x and above (https://www.cloudera.com/documentation/enterprise/5-10-x/topics/cm_metrics_impala_daemon.html). I'm using v5.10, however, when I try to do the Chart Build in CM I can only find the previous 2 metrics though (ie, "tcmalloc_physical_bytes_reserved_across_impalads" and "mem_tracker_process_limit_across_impalads" ). Also from the CM rest api, it seems like those 2 sets of metrics are the same but just different data aggregation format in the returned json? Regards, S.
... View more