question Re: YARN Capacity Scheduler Queue utilisation graph. in Archives of Support Questions (Read Only)

YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Wed, 09 Dec 2015 22:38:40 GMT

Is there any tool which will give Queue utilisation(memory, cores) graph?

I want historical data in graphical representation, so that I can adjust my workload(reschedule jobs) as per resource utilization trend over time.

Re: YARN Capacity Scheduler Queue utilisation graph.

nsabharwal — Wed, 09 Dec 2015 22:40:41 GMT

@Rahul Pathak That will be RM UI, You can see all the jobs running and using different queues.

Click Yarn in ambari --> Quick links

Re: YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Wed, 09 Dec 2015 22:44:07 GMT

I want historical data in graphical representation, so that I can adjust my workload(reschedule jobs) as per resource utilization trend over time.

Re: YARN Capacity Scheduler Queue utilisation graph.

nsabharwal — Wed, 09 Dec 2015 23:20:51 GMT

@Rahul Pathak You are looking for something like this

Re: YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Wed, 09 Dec 2015 23:34:52 GMT

@Neeraj Sabharwal I need something like this. But working on real data.

Re: YARN Capacity Scheduler Queue utilisation graph.

hemant_dindi — Mon, 19 Aug 2019 12:43:11 GMT

@Rahul Pathak

I guess Grafana with Ambari can help on this. Please use link to setup and configure Grafana.

Once Grafana is accessible, design your dashboard. Below are few sample screenshots:

Re: YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Wed, 16 Dec 2015 13:50:51 GMT

Thanks.

I will check and let you know.

Re: YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Wed, 16 Dec 2015 16:04:47 GMT

It is working fine. Thanks once again.

However I guess this is not available for ambari without ambari metrics collector.

Is there way to achieve this for Ambari Version1.5.1.110

Re: YARN Capacity Scheduler Queue utilisation graph.

hemant_dindi — Thu, 17 Dec 2015 00:19:00 GMT

For Ambari 1.5, Am not sure which fits this requirement. Will let you know, If I find anything.

Re: YARN Capacity Scheduler Queue utilisation graph.

SK1 — Wed, 10 Feb 2016 20:51:28 GMT

@Rahul Pathak @Neeraj Sabharwal : I am trying to configure grafan with ambari and trying to create graph but it is throwing me an error. So can you please point me to solution.

Problem! The requested resource doesn't exist: ServiceComponentHost not found, clusterName=HDPTST, serviceName=AMBARI_METRICS, serviceComponentName=METRICS_COLLECTOR, hostName=test.com

Re: YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Wed, 10 Feb 2016 20:59:23 GMT

@Saurabh Kumar

Ensure that you select values from drop-down list. You don't need to add values manually. Also ensure that you don't miss any value

Re: YARN Capacity Scheduler Queue utilisation graph.

SK1 — Mon, 19 Aug 2019 12:42:56 GMT

HI @Rahul Pathak I am selecting from drop down only as below . but it is giving me same error.

Re: YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Wed, 10 Feb 2016 21:16:24 GMT

What is your ambari and HDP version. Try generating graph for some other component like Resourcemanager. Also try deleting stale hosts/services from ambari.

Re: YARN Capacity Scheduler Queue utilisation graph.

SK1 — Mon, 19 Aug 2019 12:42:42 GMT

@Rahul Pathak

I have ambari Version2.2.0.0 and hdp stack 2.3.4.0.

I tried for other metrics for Resourcemanager but getting some weird error.

Problem! Unable to compile query predicate: Invalid Query Token: token='(', previous token type=VALUE_OPERAND.

Re: YARN Capacity Scheduler Queue utilisation graph.

rahulpathak109 — Thu, 11 Feb 2016 22:14:02 GMT

@Saurabh Kumar

Metrics for which you are trying to generate the graph needs to be edited manually.

Enter "metrics/yarn/Queue/root/AvailableMB" in metric box while selecting Resourcemanager

Similarly for child queue of root viz child1 you have to enter metrics/yarn/Queue/root/child1/AvailableMB

I have tested this on ambari Version 2.2.0.0 with hdp stack 2.3.4.0.