Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎06-13-2017
Accepted Solution

How can I get some monitoring data from cloudera manager(CM)?

If I want to get some monitor data from the CM, do I need to use tsquery? How can I user tsquery? Can you give me an example of using tsquery?
Posts: 390
Topics: 11
Kudos: 60
Solutions: 35
Registered: ‎09-02-2016

Re: How can I get some monitoring data from cloudera manager(CM)?

@littlewolf

 

Syntax: SELECT [metric expression] WHERE [predicate]

 

To start with tsquery, you can click on any existing charts in CM, it will show you the output based on tsquery. In general, it won't allow you to alter the existing chart and it is not recommended. So you can go to CM -> Charts -> Chart builder -> follow the above syntax, it will automatically pop-up metric expression and predicate.

 

Ex: SELECT physical_memory_used 


https://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_dg_tsquery.html

New Contributor
Posts: 2
Registered: ‎06-13-2017

Re: How can I get some monitoring data from cloudera manager(CM)?

If I want to call these data in my program, can you give me an example of calling with Java API? You can use this physical_memory example,how can I get this data in my program. I'm a new hand and can give me detailed example about this? Thank You very much.
Posts: 390
Topics: 11
Kudos: 60
Solutions: 35
Registered: ‎09-02-2016

Re: How can I get some monitoring data from cloudera manager(CM)?

Posts: 642
Topics: 3
Kudos: 105
Solutions: 67
Registered: ‎08-16-2016

Re: How can I get some monitoring data from cloudera manager(CM)?

@littlewolf you should try to use the CM API, it has a Java api.

I don't know the specific metric you are looking for but it may be there. I have used these to gather all queries issues to Hive and Impala to due some usage analysis. If it isn't in CM I would look at collecting it from the source. Most of the services and components expose metrics through JMX and host stats can be collected through some of the typical methods.

https://cloudera.github.io/cm_api/
Expert Contributor
Posts: 263
Registered: ‎01-25-2017

Re: How can I get some monitoring data from cloudera manager(CM)?

@mbigelow Do you collect the metrics using a code or a script?

 

i'm collecting alot of metrics from Yarn histroy server API and the impala API, intersting in the specific metrics you collected regarding the hive and impala usage analytics

Posts: 642
Topics: 3
Kudos: 105
Solutions: 67
Registered: ‎08-16-2016

Re: How can I get some monitoring data from cloudera manager(CM)?

@Fawze I don't collect specific metric, yet. I make an api call to get all Hive jobs between this time and that time (same for Impala) from... This data is then crunched to provide usage analysis for these specific types of jobs.

/clusters/{clusterName}/services/{serviceName}/yarnApplications
/clusters/{clusterName}/services/{serviceName}/impalaQueries
Announcements