Skip to main content
Guru_Hexa
Guru_Hexa
Silver ⭐️
August 13, 2024
Solved

Utilization Report - Cloudera Platform

Hi All,

I need to assess a Cloudera platform for a particular application of a Bank and create a utilization report. How to approach this task. I couldn't find any sample reports for reference. I found steps to download utilization report through Cloudera manager, but how to interpret it. What does each metric mean ? -- Does the  CPU / memory utilization percentage we get in the report are w.r.t the total resources that is allocated to that particular application or the overall available resources. What is the difference between pool vs users... etc.

I haven't worked on this before. Any help is much appreciated.

Thank You.

venkatsambath
venkatsambathAnswer
Gold ⭐️⭐️⭐️
August 14, 2024

Tenants are users of the cluster. Lets say you have app teams/users named dev-a, dev-b, dev-c using the cluster,  you will be estimating what resource each team will likely consume and create queue/pool to them with appropriate capacity. 

With regards to hierarchy, root would be the top most parent (represents the overall cluster resource) and you will create leaf queues/pool configured with the min/max resources you want to allocate for individual team. Applications gets assigned to the leaf queue and utilize the respective queue resource.

 

With the utilization report you will have an idea of how much each team had used in its respective queue/pool.

"There are no details about hierarchy of the utilization report" --> Are you referring to knowing utilisation at root plus all the leaf queues? https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-cluster-utilization-report-manage-resources.html in the picture shared in this doc you will see an example of overall cluster utilization and the breakup of utilization across individual queues.

https://blog.cloudera.com/yarn-capacity-scheduler/

View in thread

4 replies

DianaTorres
Community Manager
DianaTorres
Community Manager
August 13, 2024

@Guru_Hexa Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our CDP experts @venkatsambath @abdulpasithali  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
venkatsambath
Gold ⭐️⭐️⭐️
venkatsambath
Gold ⭐️⭐️⭐️
August 13, 2024

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-cluster-utilization-report-overview.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-cluster-utilization-report-manage-resources.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-cluster-utilization-report-overview-tab.html

The docs has more information. Usually for a tenant, you will have a pool configured and submit applications on the pool and utilise resources (cpu/memory) from the pool. With utilisation report you can get a sense of how much you are utilising in the configured pool, cluster and can do capacity planning based on the trend of usage.

 

Translate
Guru_Hexa
Guru_HexaAuthor
Silver ⭐️
August 14, 2024

Thank You, Venkat. 

I have already checked all these resources. There are no details about hierarchy of the utilization report in a Enterprise level cloudera framework. Will check these resources again.

And will come up with specific questions with snapshots. 

As of now - Just clarify - What is tenant ?

Thank You

Translate
    venkatsambath
    Gold ⭐️⭐️⭐️
    venkatsambathAnswer
    Gold ⭐️⭐️⭐️
    August 14, 2024

    Tenants are users of the cluster. Lets say you have app teams/users named dev-a, dev-b, dev-c using the cluster,  you will be estimating what resource each team will likely consume and create queue/pool to them with appropriate capacity. 

    With regards to hierarchy, root would be the top most parent (represents the overall cluster resource) and you will create leaf queues/pool configured with the min/max resources you want to allocate for individual team. Applications gets assigned to the leaf queue and utilize the respective queue resource.

     

    With the utilization report you will have an idea of how much each team had used in its respective queue/pool.

    "There are no details about hierarchy of the utilization report" --> Are you referring to knowing utilisation at root plus all the leaf queues? https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-use-cluster-utilization-report-manage-resources.html in the picture shared in this doc you will see an example of overall cluster utilization and the breakup of utilization across individual queues.

    https://blog.cloudera.com/yarn-capacity-scheduler/

    Translate
      Terms & ConditionsAccessibility statement