Support Questions
Find answers, ask questions, and share your expertise

How to get user level metrics for Hive or YARN?

Contributor

Is there any way to currently get user-level metrics for YARN jobs? I was hoping to get total amount of jobs submitted by the user and total amount of job failures.

I am also wanting to get user-level metrics for Hive queries for the same thing, total amount of queries and failures.

I was able to find information on getting user-level HDFS metrics but am struggling with these two other services.

1 REPLY 1

@Josh Nicholson

If you are storing your Ranger Hive logs to HDFS and running doAs=false, you can build a hive table on to of ranger-hive logs and start querying.

Example:

select requser,count(*) from ranger_audit_event_json_tmp where TO_DATE(evttime)>='2018-05-10' group by requser;

Above query gives you number queries ran by with respective to each user from 2018-05-10.

Couple of links for creating tables:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-operations/content/amb_infra_arch...

https://community.hortonworks.com/articles/60802/ranger-audit-in-hive-table-a-sample-approach-1.html

This worked for me, hope this helps you

; ;