Support Questions
Find answers, ask questions, and share your expertise

How to find complete Hive Query for application_id?

New Contributor

Hello ,

As as Hadoop admin , where we can find the complete Hive query for any application in yarn?

We can see initial 50 character of query only on RM UI.

Thanks..

4 REPLIES 4

Re: How to find complete Hive Query for application_id?

Super Guru

@Ashish Wadnerkar

To get full query running for the applicationid goto TEZ ui from ambari(there you can see query history)

Steps go to tez view via ambari:-

1) From the Ambari home page, hover over the top right corner, and select "Tez View"

2) Next, you can either search by application ID or the hive query itself to find your application.

3) Select your application - the entire hive query should be displayed here and also you can see the status of the query.

(or) from hiveserver2.log

Re: How to find complete Hive Query for application_id?

New Contributor

Thanks Shu..

One thing I forgot to mentioned that ..My job in is MR mode.

Another thing is that I tried to find the complete query in hiveserver2.log , but I didn't get it.

Could you please let me know , where I can get the info.

Re: How to find complete Hive Query for application_id?

Contributor
@Ashish Wadnerkar

You can grep hiveserver2.log with "Parsing command" string and you will get complete hive query.

Re: How to find complete Hive Query for application_id?

New Contributor

We are using cloudera version CDH-5.12.0-1.cdh5.12.0.p0.29. And we are able to parse the query in hiveServer2.logs with ": Compiling command" or ": Executing command"
e.g.

2020-02-20 09:37:23,024 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Handler-Pool: Thread-17501]: Executing command(queryId=${ACTUAL_HIVE_QUERY_ID}): ${ACTUAL_QUERY}

2020-02-20 09:37:32,590 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Handler-Pool: Thread-17332]: Compiling command(queryId=${ACTUAL_HIVE_QUERY_ID}):${ACTUAL_QUERY}

But what you need to take care of is, you need to aggregate all following lines to above line untill you find out time stamp at starting of logs. So basically it may happen that query contain multiline and in hiveServer2.log also it printed in multiline. So If you want full query then you might need to aggregate lines of logs.

 

We are actually parsing hiveServer2.log to parse all information about, query submitted by which user, what is query, what are the yarnjobs spawned to execute query, how much time each job took and Status of each yarnJob and query.