Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CodeGen Optimization time slowleness

Highlighted

CodeGen Optimization time slowleness

Explorer

Hello Folks, 

 

we are running into an interesting issue. When running queries sequentially (at 2.2.3) a query perfromance is ok, but when running several queries in parallel, sometimes the OptimizationTime withing the CodeGen section in profiles goes quite high. Do you have any ideas where the issue is or where should we look for more details?

 

The pieces of the plan below are from the same query run 2 times (sequentially and in parallel). 

 

 

Slow run

 

 CodeGen:(Total: 13s596ms, non-child: 13s596ms, % non-child: 100.00%)

         - CodegenTime: 7.30ms

         - CompileTime: 1s421ms

         - LoadTime: 148.589us

         - ModuleFileSize: 2.20 MB (2309684)

         - OptimizationTime: 11s819ms

         - PrepareTime: 351.911ms

      DataStreamSender (dst_id=4):(Total: 246.472us, non-child: 246.472us, % non-child: 100.00%)

         - BytesSent: 7.10 KB (7270)

         - NetworkThroughput(*): 1.60 MB/sec

         - OverallThroughput: 29.46 MB/sec

         - PeakMemoryUsage: 255.75 KB (261888)

         - SerializeBatchTime: 234.135us

         - ThriftTransmitTime(*): 10.984ms

         - UncompressedRowBatchSize: 10.73 KB (10984)

 

Fast run

 

 CodeGen:(Total: 2s500ms, non-child: 2s500ms, % non-child: 100.00%)

         - CodegenTime: 4.428ms

         - CompileTime: 898.891ms

         - LoadTime: 151.80us

         - ModuleFileSize: 2.20 MB (2309684)

         - OptimizationTime: 1s313ms

         - PrepareTime: 285.996ms

      DataStreamSender (dst_id=4):(Total: 184.962us, non-child: 184.962us, % non-child: 100.00%)

         - BytesSent: 7.10 KB (7269)

         - NetworkThroughput(*): 1.55 MB/sec

         - OverallThroughput: 38.12 MB/sec

         - PeakMemoryUsage: 255.75 KB (261888)

         - SerializeBatchTime: 220.153us

         - ThriftTransmitTime(*): 6.367ms

         - UncompressedRowBatchSize: 10.73 KB (10984)

 

 

Thanks