Tez query profiling

[#1]

Can someone explain why the compilation time is so big and also why we don't have other info in the dag timeline?
Also if you can explain what exactly happens when the dag is build, submitted and is running.

[#2]

@Sunil1359  

Compilation might be higher if the table has a large number of partitions or if the HMS process is slow when the query runs.  Please check the below on the corresponding time period to find the root cause. 

HS2 log
HMS log
HMS jstack 

In Tez engine queries will run in the form of DAG.  In the compilation phase, once the semantic analysis process is completed, the plan will be generated depending on the query you submitted. explain <your query> gives the plan of the query. 

Once the plan is generated DAG will be submitted to yarn and the DAG will run depending on the plan. 

As part of DAG, Split generation, input file read, shuffle fetch ..etc will be taken care and the end result will be transferred to the client.