Yarn Que Utilization Report

[#1]

Via ambari or resource manager UI I need to determine how a yarn queue is utilizing resources (CPU, Ram, disk, etc) for any given time period. If a job is running it can easily tell how the que is doing. However if i want to go back 2 weeks or 2 months, how can I do this?

[#2]

@Sunile Manjee there's LinkedIn rewinder Link and engineering had mentioned we have something the works for future release but nothing outside the box now

[#3]

@Sunile Manjee

This project looks really impressive. BTW, Great question and there is a chance that I may open it as enhancement request. https://github.com/linkedin/gobblin/wiki/Job%20Execution%20History%20Store

[#4]

I found a solution through ambari 2.2.1 which works well. Using the widgets I select AM-resources-used / AM resource available. bam! there you go. you have your que utilization.

[#5]

Hi @Sunile Manjee

If you could publish an article on this then it will be very helpful.

[#6]

Not a bad idea. Let me work on it.

[#7 · Accepted solution]

@Neeraj Sabharwal

I created a widget to do this:

and then I added this widget to my yarn page.

You can view the utilization by time using the drop down:

[#8]

@Sunile Manjee Do you have access to create articles?

[#9]

@Neeraj Sabharwal

Yes I do. You think it is better off as a article?

[#10]

@Sunile Manjee Yeah, Boss 🙂 You get 15 points for articles and we can reference your article to other users in future posts. I see that you have lots of great answers so I do recommend to post articles.