How to find complete Hive Query for application_id?

[#1]

Hello ,

As as Hadoop admin , where we can find the complete Hive query for any application in yarn?

We can see initial 50 character of query only on RM UI.

Thanks..

[#2]

@Ashish Wadnerkar

To get full query running for the applicationid goto TEZ ui from ambari(there you can see query history)

Steps go to tez view via ambari:-

1) From the Ambari home page, hover over the top right corner, and select "Tez View"

2) Next, you can either search by application ID or the hive query itself to find your application.

3) Select your application - the entire hive query should be displayed here and also you can see the status of the query.

(or) from hiveserver2.log

[#3]

Thanks Shu..

One thing I forgot to mentioned that ..My job in is MR mode.

Another thing is that I tried to find the complete query in hiveserver2.log , but I didn't get it.

Could you please let me know , where I can get the info.

[#4]

@Ashish Wadnerkar

You can grep hiveserver2.log with "Parsing command" string and you will get complete hive query.

[#5]

We are using cloudera version CDH-5.12.0-1.cdh5.12.0.p0.29. And we are able to parse the query in hiveServer2.logs with ": Compiling command" or ": Executing command"
e.g.

2020-02-20 09:37:23,024 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Handler-Pool: Thread-17501]: Executing command(queryId=${ACTUAL_HIVE_QUERY_ID}): ${ACTUAL_QUERY}

2020-02-20 09:37:32,590 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Handler-Pool: Thread-17332]: Compiling command(queryId=${ACTUAL_HIVE_QUERY_ID}):${ACTUAL_QUERY}

But what you need to take care of is, you need to aggregate all following lines to above line untill you find out time stamp at starting of logs. So basically it may happen that query contain multiline and in hiveServer2.log also it printed in multiline. So If you want full query then you might need to aggregate lines of logs.

We are actually parsing hiveServer2.log to parse all information about, query submitted by which user, what is query, what are the yarnjobs spawned to execute query, how much time each job took and Status of each yarnJob and query.

[#6]

This is not working for spark on hive. 

no applicaiton id found when grep query id.