Utilization Report - Cloudera Platform

[#1]

Hi All,

I need to assess a Cloudera platform for a particular application of a Bank and create a utilization report. How to approach this task. I couldn't find any sample reports for reference. I found steps to download utilization report through Cloudera manager, but how to interpret it. What does each metric mean ? -- Does the  CPU / memory utilization percentage we get in the report are w.r.t the total resources that is allocated to that particular application or the overall available resources. What is the difference between pool vs users... etc.

I haven't worked on this before. Any help is much appreciated.

Thank You.

[#2]

@Guru_Hexa Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our CDP experts @venkatsambath @abdulpasithali  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-cluster-...

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-clus...

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-cluster-...

The docs has more information. Usually for a tenant, you will have a pool configured and submit applications on the pool and utilise resources (cpu/memory) from the pool. With utilisation report you can get a sense of how much you are utilising in the configured pool, cluster and can do capacity planning based on the trend of usage.

[#4]

Thank You, Venkat. 

I have already checked all these resources. There are no details about hierarchy of the utilization report in a Enterprise level cloudera framework. Will check these resources again.

And will come up with specific questions with snapshots. 

As of now - Just clarify - What is tenant ?

Thank You

[#5 · Accepted solution]

Tenants are users of the cluster. Lets say you have app teams/users named dev-a, dev-b, dev-c using the cluster,  you will be estimating what resource each team will likely consume and create queue/pool to them with appropriate capacity. 

With regards to hierarchy, root would be the top most parent (represents the overall cluster resource) and you will create leaf queues/pool configured with the min/max resources you want to allocate for individual team. Applications gets assigned to the leaf queue and utilize the respective queue resource.

With the utilization report you will have an idea of how much each team had used in its respective queue/pool.

"There are no details about hierarchy of the utilization report" --> Are you referring to knowing utilisation at root plus all the leaf queues? https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-clus... in the picture shared in this doc you will see an example of overall cluster utilization and the breakup of utilization across individual queues.

https://blog.cloudera.com/yarn-capacity-scheduler/