Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Utilization Report - Cloudera Platform

avatar
New Contributor

Hi All,

I need to assess a Cloudera platform for a particular application of a Bank and create a utilization report. How to approach this task. I couldn't find any sample reports for reference. I found steps to download utilization report through Cloudera manager, but how to interpret it. What does each metric mean ? -- Does the  CPU / memory utilization percentage we get in the report are w.r.t the total resources that is allocated to that particular application or the overall available resources. What is the difference between pool vs users... etc.

I haven't worked on this before. Any help is much appreciated.

Thank You.

1 ACCEPTED SOLUTION

avatar

Tenants are users of the cluster. Lets say you have app teams/users named dev-a, dev-b, dev-c using the cluster,  you will be estimating what resource each team will likely consume and create queue/pool to them with appropriate capacity. 

With regards to hierarchy, root would be the top most parent (represents the overall cluster resource) and you will create leaf queues/pool configured with the min/max resources you want to allocate for individual team. Applications gets assigned to the leaf queue and utilize the respective queue resource.

 

With the utilization report you will have an idea of how much each team had used in its respective queue/pool.

"There are no details about hierarchy of the utilization report" --> Are you referring to knowing utilisation at root plus all the leaf queues? https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-clus... in the picture shared in this doc you will see an example of overall cluster utilization and the breakup of utilization across individual queues.

https://blog.cloudera.com/yarn-capacity-scheduler/

View solution in original post

4 REPLIES 4

avatar
Community Manager

@Guru_Hexa Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our CDP experts @venkatsambath @abdulpasithali  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-cluster-...

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-clus...

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-cluster-...

The docs has more information. Usually for a tenant, you will have a pool configured and submit applications on the pool and utilise resources (cpu/memory) from the pool. With utilisation report you can get a sense of how much you are utilising in the configured pool, cluster and can do capacity planning based on the trend of usage.

 

avatar
New Contributor

Thank You, Venkat. 

I have already checked all these resources. There are no details about hierarchy of the utilization report in a Enterprise level cloudera framework. Will check these resources again.

And will come up with specific questions with snapshots. 

As of now - Just clarify - What is tenant ?

Thank You

avatar

Tenants are users of the cluster. Lets say you have app teams/users named dev-a, dev-b, dev-c using the cluster,  you will be estimating what resource each team will likely consume and create queue/pool to them with appropriate capacity. 

With regards to hierarchy, root would be the top most parent (represents the overall cluster resource) and you will create leaf queues/pool configured with the min/max resources you want to allocate for individual team. Applications gets assigned to the leaf queue and utilize the respective queue resource.

 

With the utilization report you will have an idea of how much each team had used in its respective queue/pool.

"There are no details about hierarchy of the utilization report" --> Are you referring to knowing utilisation at root plus all the leaf queues? https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-clus... in the picture shared in this doc you will see an example of overall cluster utilization and the breakup of utilization across individual queues.

https://blog.cloudera.com/yarn-capacity-scheduler/