Support Questions
Find answers, ask questions, and share your expertise

How to get user level metrics for Hive or YARN?

How to get user level metrics for Hive or YARN?

Contributor

Is there any way to currently get user-level metrics for YARN jobs? I was hoping to get total amount of jobs submitted by the user and total amount of job failures.

I am also wanting to get user-level metrics for Hive queries for the same thing, total amount of queries and failures.

I was able to find information on getting user-level HDFS metrics but am struggling with these two other services.

1 REPLY 1

Re: How to get user level metrics for Hive or YARN?

@Josh Nicholson

If you are storing your Ranger Hive logs to HDFS and running doAs=false, you can build a hive table on to of ranger-hive logs and start querying.

Example:

select requser,count(*) from ranger_audit_event_json_tmp where TO_DATE(evttime)>='2018-05-10' group by requser;

Above query gives you number queries ran by with respective to each user from 2018-05-10.

Couple of links for creating tables:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-operations/content/amb_infra_arch...

https://community.hortonworks.com/articles/60802/ranger-audit-in-hive-table-a-sample-approach-1.html

This worked for me, hope this helps you