question Re: Obtain the actual query run on Hive when using hive.execution.engine=mr in Archives of Support Questions (Read Only)

Obtain the actual query run on Hive when using hive.execution.engine=mr

Mark_Petronic — Wed, 07 Sep 2016 03:22:15 GMT

As an administrator, you can easily use the Tez view in Ambari to see the actual full HQL query that was executed by any user. How do you obtain the actual query that was run when hive.execution.engine=mr?

Re: Obtain the actual query run on Hive when using hive.execution.engine=mr

cstanca — Wed, 07 Sep 2016 08:31:09 GMT

@Mark Petronic

One option is to monitor the hive log located at /var/log/hive

I usually run a continuous tail and grep for keywords specific for SQL (you can build a file and read those keywords from the file and then grep, use a few pipes to achieve this)

tail -f hiveserver2.log | grep SELECT FROM

or run post-mortem a cat command with grep on the same file knowing the approximate timestamps. One can build an intelligent script to parse the logs if each query executed starts and ends with setting a sort of tag variable.

Unfortunately, this is a bit ugly. That's where HiveSQL was before Tez.

If any of the responses to your question addressed the problem don't forget to vote and accept the answer. If you fix the issue on your own, don't forget to post the answer to your own question. A moderator will review it and accept it.

Re: Obtain the actual query run on Hive when using hive.execution.engine=mr

Mark_Petronic — Sat, 10 Sep 2016 05:18:59 GMT

Thanks @Constantin Stanca. Parsing that log will work just fine for my use case. Appreciate your help!!!