11-17-2017 02:00 PM
In Cloudera Mgmt Service, Activity Monitor is configured to write to Oracle database and retain data for 7 days.
I can see the data in the dashboard for various metrics, but when I look in the database, the tables e.g. cdh_monitor.cmon_act_lmv_2017_11_17 are being created, but there is no data in the tables.
Not sure why data is not being stored in these tables. And, what's the source of data for charts displayed on the dashboard?
Appreciate any help!
11-19-2017 03:56 AM
Activity Monitor is only used if you have MapReduce1 in your cluster. If you are utilizing only YARN for your jobs, you can remove Activity Monitor without impact as it will not be doing anything
11-19-2017 04:32 AM - edited 11-19-2017 04:36 AM
Thank you for your response, Ben.
Could you point me to the source of data where chart builder fetches from?
e.g. this query for Cluster CPU:
SELECT cpu_percent_across_hosts WHERE entityName = "1" AND category = CLUSTER
or this one for HDFS IO:
SELECT total_bytes_written_rate_across_datanodes WHERE entityName = "hdfs:nameservice" AND category = SERVICE
These don't seem to be standard SQL stmts as these don't include the table name and even the "FROM" clause. I am looking to access the raw data behind these metrics and potentially gather data for additional custom metrics that I could define on clients' requests.
11-19-2017 05:01 AM
Check out this documentation for some background on where the charts data comes from:
metrics are stored in the Host Monitor and Service Monitor. The data used in chart building is avaiable via the tsquery results. The actual data is stored in leveldb files on disk (by default in /var/lib/cloudera-host-monitor and /var/lib/cloudera-service-monitor)
If you wish to view the metrics in JSON format you can click on the properties icon in the chart and choose "Export JSON"