I'm trying to test Hive queries running on the cluster. When I execute 10 simple queries (using Ambari Hive Views / TEZ), I can see them all running in YARN.
However in Ambari Hive View, they finish in 1 minute or so, but it takes a further 4-6 minutes based on the YARN Finish Time for it to be marked with a Final Status of 'Succeeded' in YARN before those resources are allocated to the next queries in line.
After doing some research, I found that it might come down to:
It looks like the tez.session.am.dag.submit.timeout.secs is the one that might be impacting me as it sounds like there is a 300s delay (~5 minutes) which is consistent to what I'm saying. However this seems to be the default that is suggested by Hortonworks and other users?
Is there any other setting I can look at tweaking?
Could the tez.session.am.dag.submit.timeout.secs be changed to something like 30 seconds to allow the next query to run - or are there implications to this?
Answering an old question, not sure if it's already resolved :)
tez.am.container.reuse.enabled @Lucy Trengel , can you tell if it's set to true ? As this will help you in specifying whether container should be reused !
for my case, tez.am.container.reuse.enabled = True. I ran a simple select * statement from the readonly user, but still the Tez session is in running state for the user.
application_1560323362594_1913 HIVE-9c2f473e-a685-4233-a522-394933d2905d TEZ readonly default RUNNING UNDEFINED 0% http://mydomain:37441/ui/
Can any one tell me what was the issue. How to use Tez to use the resource correctly. Please dont share any document, I am tired. Please post me practical solution.