Utility to for utilization.

[#1]

Hi

Is there any utility to find out the resource utilization for particular job level.

Like:- CPU, memory, per queue utilization, time utilization.

[#2 · Accepted solution]

1. If you're using Tez, it has a really nice view for those kinds of statistics. Ambari provides access to the view in a secure way.

2. SmartSense is an offering for Hortonworks customers and leverages Ambari Metrics Collector to provide you with metrics across Hive, HBase, HDFS, Spark, YARN, etc. Including capacity projections months ahead. It is provided via Apache Spark queries and Apache Zeppelin notebook.

3. You can look elsewhere like Dr. Elephant to build your own solution

I'm biased but SmartSense I believe gives the best overall view of a cluster. If you are a customer, I don't see a reason not to use it.

[#3]

@Artem Ervits

Is there any docs link to go through for more details?

And this will be provide me the per job utilization details or overall cluster level detail?

[#4]

@Ashnee Sharma here you go http://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.3.1/bk_user-guide/content/activity_explore...

go through each page, it says specifically the level of detail per component. In the next release, we're going to add cluster capacity projections and more extensive analysis of components.

If this answers your question, please mark accept the answer as best.