Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Report on Compute - CPU and Memory - by project or tenant?

Solved Go to solution

Report on Compute - CPU and Memory - by project or tenant?

Contributor

Hello -

 

We need to be able to report back on cluster usage.  I can use the existing reports in CM to figure out the HDFS (or storage) usage for our projects and customers.  However, I also need a way to calculate and report on the compute resources used per project or customer/tenant.

 

Does anyone know how to do this? I have heard it is possible, but I have searched and haven't found anything.

 

Thanks for any help or suggestions

...

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Report on Compute - CPU and Memory - by project or tenant?

Master Collaborator

For Mapreduce and Spark jobs (running on YARN) you should be able to report from Resource Pools.

There is a chart of Per Pool allocation, containers running etc. 

Of course Impala usage is not included, there you have to report from Impala Queries Workload summary and manually report from the history of queries - you have to choose a KPI, whether it is cpu time, or HDFS scanned or something else.

 

 

2 REPLIES 2

Re: Report on Compute - CPU and Memory - by project or tenant?

Master Collaborator

For Mapreduce and Spark jobs (running on YARN) you should be able to report from Resource Pools.

There is a chart of Per Pool allocation, containers running etc. 

Of course Impala usage is not included, there you have to report from Impala Queries Workload summary and manually report from the history of queries - you have to choose a KPI, whether it is cpu time, or HDFS scanned or something else.

 

 

Highlighted

Re: Report on Compute - CPU and Memory - by project or tenant?

Contributor

Thanks for the reply and information.

 

We do have Resource Pools set up now...but more by function.  However...from what you are saying (and from the few things I have seen), the only way for us to get what we want is to create resource pools per project/customer.

 

I forgot about the Impala stuff....about them not being included.  Thanks for mentioning that too.

 

thank you!

 

Don't have an account?
Coming from Hortonworks? Activate your account here