Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Understanding Impala operator time

Understanding Impala operator time

Explorer

Hello,

 

I ran an Impala query and tried to understand some time metrics for operators. In the exe summary, I have following:

 

    ExecSummary: 
Operator              #Hosts   Avg Time   Max Time    #Rows  Est. #Rows   Peak Mem  Est. Peak Mem  Detail                         
----------------------------------------------------------------------------------------------------------------------------------
....
06:HASH JOIN               1    1s051ms    1s051ms  857.79K          -1    8.66 MB      266.80 KB  INNER JOIN, BROADCAST          
|--11:EXCHANGE             1   11.914us   11.914us       61       7.30K          0              0  BROADCAST                      
|  03:SCAN HDFS            1   83.745ms   83.745ms       61       7.30K   18.05 MB       48.00 MB  tpcds_text_100.date_dim        
...

However, when I look into the fragment which has the Operator information, I see the following:

 

HASH_JOIN_NODE (id=6):(Total: 2m42s, non-child: 1s051ms, % non-child: 0.65%)

 

My question is, how could the total time of that operator being 2m42s while the avg/max time of the operator being only 1s51ms?

 

For complete Exec summary of this query, please see below:

 

    ExecSummary: 
Operator              #Hosts   Avg Time   Max Time    #Rows  Est. #Rows   Peak Mem  Est. Peak Mem  Detail                         
----------------------------------------------------------------------------------------------------------------------------------
14:MERGING-EXCHANGE        1   46.195us   46.195us      100         100          0        -1.00 B  UNPARTITIONED                  
08:TOP-N                   1    2.566ms    2.566ms      100         100   36.00 KB        6.15 KB                                 
13:AGGREGATE               1   60.442ms   60.442ms   54.98K          -1   14.26 MB      128.00 MB  FINALIZE                       
12:EXCHANGE                1    8.420ms    8.420ms   95.31K          -1          0              0  HASH(w_warehouse_name,i_ite... 
07:AGGREGATE               1  316.838ms  316.838ms   95.31K          -1   19.12 MB      128.00 MB  STREAMING                      
06:HASH JOIN               1    1s051ms    1s051ms  857.79K          -1    8.66 MB      266.80 KB  INNER JOIN, BROADCAST          
|--11:EXCHANGE             1   11.914us   11.914us       61       7.30K          0              0  BROADCAST                      
|  03:SCAN HDFS            1   83.745ms   83.745ms       61       7.30K   18.05 MB       48.00 MB  tpcds_text_100.date_dim        
05:HASH JOIN               1    3s907ms    3s907ms   25.08M          -1    8.40 MB        2.00 GB  INNER JOIN, BROADCAST          
|--10:EXCHANGE             1   11.369us   11.369us       15          -1          0              0  BROADCAST                      
|  01:SCAN HDFS            1   40.714ms   40.714ms       15          -1   41.00 KB       32.00 MB  tpcds_text_100.warehouse       
04:HASH JOIN               1      1m53s      1m53s   25.08M          -1   22.40 GB        2.00 GB  INNER JOIN, BROADCAST          
|--09:EXCHANGE             1   12s011ms   12s011ms  399.33M          -1          0              0  BROADCAST                      
|  00:SCAN HDFS            3    1s583ms    1s608ms  399.33M          -1  514.87 MB        3.44 GB  tpcds_text_100.inventory       
02:SCAN HDFS               1   43.921ms   43.921ms   12.86K      20.40K   64.67 MB      128.00 MB  tpcds_text_100.item 

 

Thanks,

S.