Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Capacity Scheduler statistics

Solved Go to solution

Capacity Scheduler statistics

Hi,

Is there any way to get statistics for the usage of the queue of the cluster for a period of a month. I have configured 3 groups A, B & C in allocating A=30%, B=50% and C=20% now I would like to get statics like when is the queue heavily used, which is not and at a particular time I would like to know what is the percentage queue it used, which are peak hour's which is using its queue beyond 100% for particular queue etc.,

Is there any command above information for the capacity scheduler. Thanks in advance.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Capacity Scheduler statistics

@Sri Bandaru

I think there is no direct way to achieve it untill you store metrics somewhere and have query over it. you can poll the RM scheduler REST API (http://<rm http address:port>/ws/v1/cluster/scheduler) and store the target queue usage somewhere and then run query over it.

3 REPLIES 3

Re: Capacity Scheduler statistics

@Sri Bandaru

I think there is no direct way to achieve it untill you store metrics somewhere and have query over it. you can poll the RM scheduler REST API (http://<rm http address:port>/ws/v1/cluster/scheduler) and store the target queue usage somewhere and then run query over it.

Re: Capacity Scheduler statistics

@Rajkumar Singh

Thank's for the response, I need the data of queue information for the last one month to get the metrics.

Re: Capacity Scheduler statistics

Contributor

@Sri Bandaru, you can get the queue related metrics through AMS. AMS has various queue related metrics like ActiveApplications, AggregateContainersAllocated, AggregateContainersReleased, AllocatedContainers, AllocatedMB, AllocatedVCores, AppsPending, AppsSubmitted, AvailableMB, AvailableVCores, PendingContainers, PendingMB, PendingVCores, ReservedContainers, ReservedMB, ReservedVCores, running_0, running_60, running_300, running_1440 etc. This will provdie you what your are looking for. Also if you are on Ambari 2.2.2+ you will have graphana UI in AMS where you can create your dashboards for the metrics of your interest.