Estimate run time of hive query

[#1]

Is there anyway to find out how long does a hive query take to get completed?

[#2 · Accepted solution]

Well, there is no way to know in advance how long will a query you've never tried before take on your cluster. Besides query complexity and resource requirements it also depends on the size and occupancy of the Yarn queue you are using. Even the same query can take longer the first time to start Tez containers if not already started, or to load data to memory in case of LLAP. You can try to run your representative queries and measure their execution time, after that you will roughly know how long are queries from the same class going to take (again if Yarn queue conditions are about the same).

[#3]

Thanks Predrag Minovic. One more question is there way to check the what are the queries executed through ambari or admin console?

[#4]

Sorry for the delay, the best way to check your queries provided it's Hive on Tez, which is pretty much standard these days since it's much faster than Hive on MR, is through Ambari Tez view. You can see the query, start and end time, and DAG details, counters, graphical view of used mappers and reducers, indeed very detailed information. Note that if you run Pig on Tez those queries will appear in the Tez view as well. Give it a try!

[#5]

Hi @Bala Vignesh N V, please consider accepting my answer to help us manage answered questions. Tnx!