Capacity Scheduler statistics

[#1]

Hi,

Is there any way to get statistics for the usage of the queue of the cluster for a period of a month. I have configured 3 groups A, B & C in allocating A=30%, B=50% and C=20% now I would like to get statics like when is the queue heavily used, which is not and at a particular time I would like to know what is the percentage queue it used, which are peak hour's which is using its queue beyond 100% for particular queue etc.,

Is there any command above information for the capacity scheduler. Thanks in advance.

[#2 · Accepted solution]

@Sri Bandaru

I think there is no direct way to achieve it untill you store metrics somewhere and have query over it. you can poll the RM scheduler REST API (http://<rm http address:port>/ws/v1/cluster/scheduler) and store the target queue usage somewhere and then run query over it.

[#3]

@Rajkumar Singh

Thank's for the response, I need the data of queue information for the last one month to get the metrics.

[#4]

@Sri Bandaru, you can get the queue related metrics through AMS. AMS has various queue related metrics like ActiveApplications, AggregateContainersAllocated, AggregateContainersReleased, AllocatedContainers, AllocatedMB, AllocatedVCores, AppsPending, AppsSubmitted, AvailableMB, AvailableVCores, PendingContainers, PendingMB, PendingVCores, ReservedContainers, ReservedMB, ReservedVCores, running_0, running_60, running_300, running_1440 etc. This will provdie you what your are looking for. Also if you are on Ambari 2.2.2+ you will have graphana UI in AMS where you can create your dashboards for the metrics of your interest.