Our team working on Hadoop framework/technologies with Hive, Ambari, Ranger and planning to create a dashboard, which provides following information :
For now, we are trying to use HiveMetastore and Ranger audit logs to access the above mentioned information.
Is there any other better way to fetch the information above ?
Kindly let me know, if I need to provide any more information.
You might want to take a look at using the Tez UI, which for HDP 2.6 should have execution time, user, db/table usage, user, and resource usage. This is more focused on queries which run with the Tez execution engine. This makes use of information sent to the Yarn Timeline Server through both Hive (using the ATSHook) and Tez. You could also look into querying the Timeline Server directly for this data as well. There have been a number of updates to the information sent to ATS through Hive/Tez recently, to go into HDP 2.6