Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How can I get some monitoring data from cloudera manager(CM)?

avatar
New Contributor
If I want to get some monitor data from the CM, do I need to use tsquery? How can I user tsquery? Can you give me an example of using tsquery?
2 ACCEPTED SOLUTIONS

avatar
Champion
@littlewolf you should try to use the CM API, it has a Java api.

I don't know the specific metric you are looking for but it may be there. I have used these to gather all queries issues to Hive and Impala to due some usage analysis. If it isn't in CM I would look at collecting it from the source. Most of the services and components expose metrics through JMX and host stats can be collected through some of the typical methods.

https://cloudera.github.io/cm_api/

View solution in original post

avatar
Champion
@Fawze I don't collect specific metric, yet. I make an api call to get all Hive jobs between this time and that time (same for Impala) from... This data is then crunched to provide usage analysis for these specific types of jobs.

/clusters/{clusterName}/services/{serviceName}/yarnApplications
/clusters/{clusterName}/services/{serviceName}/impalaQueries

View solution in original post

6 REPLIES 6

avatar
Champion

@littlewolf

 

Syntax: SELECT [metric expression] WHERE [predicate]

 

To start with tsquery, you can click on any existing charts in CM, it will show you the output based on tsquery. In general, it won't allow you to alter the existing chart and it is not recommended. So you can go to CM -> Charts -> Chart builder -> follow the above syntax, it will automatically pop-up metric expression and predicate.

 

Ex: SELECT physical_memory_used 


https://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_dg_tsquery.html

avatar
New Contributor
If I want to call these data in my program, can you give me an example of calling with Java API? You can use this physical_memory example,how can I get this data in my program. I'm a new hand and can give me detailed example about this? Thank You very much.

avatar
Champion

avatar
Champion
@littlewolf you should try to use the CM API, it has a Java api.

I don't know the specific metric you are looking for but it may be there. I have used these to gather all queries issues to Hive and Impala to due some usage analysis. If it isn't in CM I would look at collecting it from the source. Most of the services and components expose metrics through JMX and host stats can be collected through some of the typical methods.

https://cloudera.github.io/cm_api/

avatar
Master Collaborator

@mbigelow Do you collect the metrics using a code or a script?

 

i'm collecting alot of metrics from Yarn histroy server API and the impala API, intersting in the specific metrics you collected regarding the hive and impala usage analytics

avatar
Champion
@Fawze I don't collect specific metric, yet. I make an api call to get all Hive jobs between this time and that time (same for Impala) from... This data is then crunched to provide usage analysis for these specific types of jobs.

/clusters/{clusterName}/services/{serviceName}/yarnApplications
/clusters/{clusterName}/services/{serviceName}/impalaQueries