question Hive Logs: View counters for "MR" execution engine in Archives of Support Questions (Read Only)

Hive Logs: View counters for "MR" execution engine

jahankk — Mon, 11 Dec 2017 16:39:19 GMT

Hi,

I wanted to get details of the counters for each phase in map-reduce when hive queries are executed via "mr" engine.

For queries executed via TEZ i m able to get the details from yarn timeline server based on the DAG id/app id of the query.http://<ip-addr>:8188/ws/v1/timeline/TEZ_DAG_ID?primaryFilter=applicationId:<app-ID>

Basically I'm interested in number of records returned by the query, I dont even need counters for each map-reduce phase.

Re: Hive Logs: View counters for "MR" execution engine

ghagleitner — Thu, 14 Dec 2017 06:27:52 GMT

For MR you should be able to get the counters from the Yarn UI for the MR job that ran as a result of the query.

Re: Hive Logs: View counters for "MR" execution engine

jahankk — Thu, 14 Dec 2017 09:44:25 GMT

hi, thanks @ghagleitner. but i tried navigating yarn ui. but could find the counters. can you please give the navingation path

Re: Hive Logs: View counters for "MR" execution engine

jahankk — Thu, 14 Dec 2017 17:07:22 GMT

Thanks - I located the counters. Btw, is there an rest api end point to access job tracker/counters information ?