I have configured graphite and grafana for monitoring the spark applications as per "https://community.hortonworks.com/articles/222813/monitoring-spark-2-performance-via-grafana-in-amba-1.html". Are the below queries, the correct ones ? Driver Memory Driver Heap Usage aliasByNode($application.driver.jvm.heap.usage, 1) Driver JVM Memory Pools Usage aliasByNode($application.driver.jvm.pools.*.used, 4) Executor & Driver Memory Used aliasByNode($application.*.jvm.heap.used, 1) Executor Memory Used aliasByNode(exclude($application.*.jvm.heap.used, '.driver.jvm.heap'), 1) alias(sumSeries(exclude($application.*.jvm.heap.used, '.driver.jvm.heap')), 'total') Task Executor Active Tasks Per Executor aliasByNode(summarize($application.*.executor.threadpool.activeTasks, '10s', 'sum', false), 1) Completed Tasks per Executor aliasByNode($application.*.executor.threadpool.completeTasks, 1) Completed Tasks/Minute per Executor aliasByNode(nonNegativeDerivative(summarize($application.*.executor.threadpool.completeTasks, '1m', 'avg', false)), 1) Read/Write IOPS Read IOPS alias(perSecond(sumSeries($application.*.executor.filesystem.hdfs.read_ops)), 'total') aliasByNode(perSecond($application.*.executor.filesystem.hdfs.read_ops), 1) Write IOPS alias(perSecond(sumSeries($application.*.executor.filesystem.hdfs.write_ops)), 'total') aliasByNode(perSecond($application.*.executor.filesystem.hdfs.write_ops), 1) HDFS Bytes Reads/Writes Per Executor Executor HDFS Reads aliasByMetric($application.*.executor.filesystem.hdfs.read_bytes) Executor HDFS Bytes Written aliasByMetric($application.*.executor.filesystem.hdfs.write_bytes Also does grafana and graphite provides metrices on the below use case ? We have a bunch of hourly / daily batches on Airflow. This batch use PySpark for data processing. We want to see historical trend of Spark memory usage on the same batch. So we want to aggregate Spark applications on the same batch then visualize historical trends so we can check if how memory usage is increased based on traffic ... View more