The job is complete, but has status running

[#1]

Hi. I've got a little problem with YARN ResourceManager UI and executing job in Hue. I execute simple query in Hue:

select ip, count(*)
from dns_data_huge_parquet
group by ip
having count(*) > 50
order by ip asc

I got results after about 10 seconds and everything looks great. But Job Browser in Hue and ResourceManager UI (YARN functionality) show that this job is still running.

The job will have got status "Succeeded" after 11-15 minutes. My question is - why application is still running when it is complete and I can see results?

[#2 · Accepted solution]

It is a Tez application. They stay around for a while to wait for new dags ( execution graph) otherwise you need to create a new session for every query which adds around 20s to your query time.

Its configured here (normally a couple minutes)

tez.session.am.dag.submit.timeout.secs