Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Obtain the actual query run on Hive when using hive.execution.engine=mr

avatar
Expert Contributor

As an administrator, you can easily use the Tez view in Ambari to see the actual full HQL query that was executed by any user. How do you obtain the actual query that was run when hive.execution.engine=mr?

1 ACCEPTED SOLUTION

avatar
Super Guru

@Mark Petronic

One option is to monitor the hive log located at /var/log/hive

I usually run a continuous tail and grep for keywords specific for SQL (you can build a file and read those keywords from the file and then grep, use a few pipes to achieve this)

tail -f hiveserver2.log | grep SELECT FROM

or run post-mortem a cat command with grep on the same file knowing the approximate timestamps. One can build an intelligent script to parse the logs if each query executed starts and ends with setting a sort of tag variable.

Unfortunately, this is a bit ugly. That's where HiveSQL was before Tez.

If any of the responses to your question addressed the problem don't forget to vote and accept the answer. If you fix the issue on your own, don't forget to post the answer to your own question. A moderator will review it and accept it.

View solution in original post

2 REPLIES 2

avatar
Super Guru

@Mark Petronic

One option is to monitor the hive log located at /var/log/hive

I usually run a continuous tail and grep for keywords specific for SQL (you can build a file and read those keywords from the file and then grep, use a few pipes to achieve this)

tail -f hiveserver2.log | grep SELECT FROM

or run post-mortem a cat command with grep on the same file knowing the approximate timestamps. One can build an intelligent script to parse the logs if each query executed starts and ends with setting a sort of tag variable.

Unfortunately, this is a bit ugly. That's where HiveSQL was before Tez.

If any of the responses to your question addressed the problem don't forget to vote and accept the answer. If you fix the issue on your own, don't forget to post the answer to your own question. A moderator will review it and accept it.

avatar
Expert Contributor

Thanks @Constantin Stanca. Parsing that log will work just fine for my use case. Appreciate your help!!!