Query (id=244b213cd61a74ca:145f38ad00000000) Summary Session ID: 45422724aac24718:12f97cc20cff7eb0 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2019-08-29 10:46:28.618999000 End Time: 2019-08-29 11:04:08.123281000 Query Type: QUERY Query State: EXCEPTION Query Status: Query 244b213cd61a74ca:145f38ad00000000 expired due to client inactivity (timeout is 10m) Impala Version: impalad version 2.11.0-cdh5.14.4 RELEASE (build 20e635646a13347800fad36a7d0b1da25ab32404) User: hue/chia.haas.berkeley.edu@CAMPUS.BERKELEY.EDU Connected User: hue/chia.haas.berkeley.edu@CAMPUS.BERKELEY.EDU Delegated User: Network Address: ::ffff:10.0.0.10:52714 Default Db: derived_v4 Sql Statement: select * from external_data.physician_compare where primary_specialty = 'INTERNAL MEDICINE' Coordinator: chia.haas.berkeley.edu:22000 Query Options (set by configuration): QUERY_TIMEOUT_S=600 Query Options (set by configuration and planner): QUERY_TIMEOUT_S=600,MT_DOP=0 Plan: ---------------- Max Per-Host Resource Reservation: Memory=0B Per-Host Resource Estimates: Memory=176.00MB WARNING: The following tables are missing relevant table and/or column statistics. external_data.physician_compare F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=0B mem-reservation=0B PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 01:EXCHANGE [UNPARTITIONED] | mem-estimate=0B mem-reservation=0B | tuple-ids=0 row-size=615B cardinality=unavailable | F00:PLAN FRAGMENT [RANDOM] hosts=5 instances=5 Per-Host Resources: mem-estimate=176.00MB mem-reservation=0B 00:SCAN HDFS [external_data.physician_compare, RANDOM] partitions=1/1 files=1 size=545.28MB predicates: primary_specialty = 'INTERNAL MEDICINE' stats-rows=unavailable extrapolated-rows=disabled table stats: rows=unavailable size=0B column stats: unavailable parquet dictionary predicates: primary_specialty = 'INTERNAL MEDICINE' mem-estimate=176.00MB mem-reservation=0B tuple-ids=0 row-size=615B cardinality=unavailable ---------------- Estimated Per-Host Mem: 184549376 Tables Missing Stats: external_data.physician_compare Per Host Min Reservation: chia.haas.berkeley.edu:22000(0) chia1.haas.berkeley.edu:22000(0) chia2.haas.berkeley.edu:22000(0) chia3.haas.berkeley.edu:22000(0) chia4.haas.berkeley.edu:22000(0) chia5.haas.berkeley.edu:22000(0) Request Pool: root.hue Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail -------------------------------------------------------------------------------------------------------------------------- 01:EXCHANGE 1 163.511ms 163.511ms 1.02K -1 0 0 UNPARTITIONED 00:SCAN HDFS 5 60.886ms 70.837ms 22.53K -1 78.33 MB 176.00 MB external_data.physician_compare Errors: - ComputeScanRangeAssignmentTimer: 107.12us (107119) - InactiveTotalTime: 0ns (0) - TotalTime: 0ns (0) Planner Timeline Analysis finished: 1ms (1046465) Value transfer graph computed: 1ms (1157401) Single node plan created: 2ms (2542104) Runtime filters computed: 2ms (2628582) Distributed plan created: 2ms (2671973) Lineage info computed: 2ms (2978924) Planning finished: 4ms (4604298) Query Timeline Query submitted: 33.06us (33061) Planning finished: 6ms (6224094) Submit for admission: 6ms (6929540) Completed admission: 7ms (7000385) Ready to start on 6 backends: 7ms (7533002) All 6 execution backends (6 fragment instances) started: 9ms (9134390) Rows available: 197ms (197327732) First row fetched: 1.48s (1479618752) Cancelled: 10.0m (601878118349) Unregister query: 17.7m (1059504305080) ImpalaServer - ClientFetchWaitTimer: 17.7m (1059305439100) - InactiveTotalTime: 0ns (0) - RowMaterializationTimer: 1ms (1243582) - TotalTime: 0ns (0) Execution Profile 244b213cd61a74ca:145f38ad00000000 Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 6, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Per Node Peak Memory Usage: chia.haas.berkeley.edu:22000(10.79 MB) chia1.haas.berkeley.edu:22000(79.99 MB) chia2.haas.berkeley.edu:22000(35.32 MB) chia3.haas.berkeley.edu:22000(76.68 MB) chia4.haas.berkeley.edu:22000(76.68 MB) chia5.haas.berkeley.edu:22000(76.68 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0ns (0) - InactiveTotalTime: 0ns (0) - NumBackends: 6 (6) - NumFragmentInstances: 6 (6) - NumFragments: 2 (2) - TotalTime: 191ms (191071590) Averaged Fragment F01 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:10m1s max:10m1s mean: 10m1s stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 10.8 MiB (11287687) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 10.8 MiB (11311906) - RowsProduced: 1,024 (1024) - TotalNetworkReceiveTime: 163ms (163478924) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 163ms (163496759) - TotalThreadsSysTime: 1.00us (1000) - TotalThreadsUserTime: 27.00us (27000) - TotalThreadsVoluntaryContextSwitches: 1 (1) - TotalTime: 188ms (188169656) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 32.46us (32465) - InactiveTotalTime: 0ns (0) - OpenTime: 163ms (163500325) - ExecTreeOpenTime: 163ms (163482604) - PrepareTime: 24ms (24618345) - ExecTreePrepareTime: 41.55us (41552) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - TotalTime: 0ns (0) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 23ms (23393775) - TotalTime: 24ms (24257446) EXCHANGE_NODE (id=1) - ConvertRowBatchTime: 30.74us (30740) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1,024 (1024) - RowsReturnedRate: 6262 per second (6262) - TotalTime: 163ms (163511666) DataStreamReceiver - BytesReceived: 5.2 MiB (5482329) - DeserializeRowBatchTimer: 18ms (18219782) - FirstBatchArrivalWaitTime: 163ms (163458747) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 10.8 MiB (11279495) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Coordinator Fragment F01 Instance 244b213cd61a74ca:145f38ad00000000 (host=chia.haas.berkeley.edu:22000) - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 10.8 MiB (11287687) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 10.8 MiB (11311906) - RowsProduced: 1,024 (1024) - TotalNetworkReceiveTime: 163ms (163478924) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 163ms (163496759) - TotalThreadsSysTime: 1.00us (1000) - TotalThreadsUserTime: 27.00us (27000) - TotalThreadsVoluntaryContextSwitches: 1 (1) - TotalTime: 188ms (188169656) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 32.46us (32465) - InactiveTotalTime: 0ns (0) - OpenTime: 163ms (163500325) - ExecTreeOpenTime: 163ms (163482604) - PrepareTime: 24ms (24618345) - ExecTreePrepareTime: 41.55us (41552) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - TotalTime: 0ns (0) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 23ms (23393775) - TotalTime: 24ms (24257446) EXCHANGE_NODE (id=1) - ConvertRowBatchTime: 30.74us (30740) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1,024 (1024) - RowsReturnedRate: 6262 per second (6262) - TotalTime: 163ms (163511666) DataStreamReceiver - BytesReceived: 5.2 MiB (5482329) - DeserializeRowBatchTimer: 18ms (18219782) - FirstBatchArrivalWaitTime: 163ms (163458747) - InactiveTotalTime: 163ms (163459743) - PeakMemoryUsage: 10.8 MiB (11279495) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Averaged Fragment F00 split sizes: min: 33.28 MB, max: 128.00 MB, avg: 109.06 MB, stddev: 37.89 MB completion times: min:10m1s max:10m1s mean: 10m1s stddev:333.270us execution rates: min:56.62 KB/sec max:217.77 KB/sec mean:185.54 KB/sec stddev:64.46 KB/sec num instances: 5 - AverageThreadTokens: 1.00 (1.0008364014439246) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 69.1 MiB (72423687) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 69.1 MiB (72423687) - RowsProduced: 4,505 (4505) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1.89us (1886) - TotalStorageWaitTime: 23ms (23013764) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 540ms (540058309) - TotalThreadsSysTime: 21ms (21401400) - TotalThreadsUserTime: 495ms (495403600) - TotalThreadsVoluntaryContextSwitches: 3 (3) - TotalTime: 150ms (150600547) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 60ms (60575642) - InactiveTotalTime: 0ns (0) - OpenTime: 121ms (121094546) - ExecTreeOpenTime: 38.91us (38908) - PrepareTime: 29ms (29453929) - ExecTreePrepareTime: 124.53us (124532) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.4 MiB (1418415) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 623.7 MiB/s (654034483) - OverallThroughput: 51.6 MiB/s (54130741) - PeakMemoryUsage: 192 B (192) - RowsReturned: 3,481 (3481) - SerializeBatchTime: 24ms (24742707) - TotalTime: 26ms (26489436) - TransmitDataRPCTime: 2ms (2183703) - UncompressedRowBatchSize: 3.9 MiB (4137675) CodeGen - CodegenTime: 5ms (5271413) - CompileTime: 35ms (35217246) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 84ms (84799803) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 26ms (26172829) - TotalTime: 147ms (147060375) HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.00 (3.350086101982625E-4) - AverageScannerThreadConcurrency: 0.00 (8.368212556812045E-4) - BytesRead: 109.1 MiB (114405988) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 67.4 MiB (70684262) - PerReadThreadRawHdfsThroughput: 791.6 MiB/s (830098759) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 407,271 (407271) - RowsReturned: 4,505 (4505) - RowsReturnedRate: 73532 per second (73532) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 418ms (418972940) - DelimiterParseTime: 339ms (339993324) - MaterializeTupleTime(*): 54ms (54217150) - ScannerThreadsSysTime: 20ms (20386200) - ScannerThreadsUserTime: 375ms (375530200) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 5ms (5969265) - TotalRawHdfsReadTime(*): 142ms (142338293) - TotalReadThroughput: 186.9 KiB/s (191389) - TotalTime: 60ms (60886750) Fragment F00 Instance 244b213cd61a74ca:145f38ad00000004 (host=chia1.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008375209380234) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 80.0 MiB (83872417) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 80.0 MiB (83872417) - RowsProduced: 3,072 (3072) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1.57us (1567) - TotalStorageWaitTime: 52ms (52633108) - TotalThreadsInvoluntaryContextSwitches: 2 (2) - TotalThreadsTotalWallClockTime: 638ms (638499595) - TotalThreadsSysTime: 27ms (27005000) - TotalThreadsUserTime: 558ms (558676000) - TotalThreadsVoluntaryContextSwitches: 4 (4) - TotalTime: 162ms (162306845) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 51ms (51049454) - InactiveTotalTime: 0ns (0) - OpenTime: 132ms (132824827) - ExecTreeOpenTime: 40.17us (40168) - PrepareTime: 29ms (29430201) - ExecTreePrepareTime: 145.72us (145718) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 943.2 KiB (965808) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 669.1 MiB/s (701634275) - OverallThroughput: 47.4 MiB/s (49710353) - PeakMemoryUsage: 192 B (192) - RowsReturned: 2,048 (2048) - SerializeBatchTime: 18ms (18604496) - TotalTime: 19ms (19428709) - TransmitDataRPCTime: 1ms (1376512) - UncompressedRowBatchSize: 2.7 MiB (2828308) CodeGen - CodegenTime: 5ms (5358267) - CompileTime: 38ms (38585458) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 93ms (93219678) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 26ms (26034233) - TotalTime: 158ms (158722014) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (8.382229673093043E-4) - AverageScannerThreadConcurrency: 0.00 (8.382229673093043E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 78.3 MiB (82132992) - PerReadThreadRawHdfsThroughput: 719.5 MiB/s (754484802) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 477,833 (477833) - RowsReturned: 3,072 (3072) - RowsReturnedRate: 59757 per second (59757) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 2 (2) - ScannerThreadsTotalWallClockTime: 505ms (505683841) - DelimiterParseTime: 389ms (389263619) - MaterializeTupleTime(*): 61ms (61827898) - ScannerThreadsSysTime: 25ms (25998000) - ScannerThreadsUserTime: 426ms (426968000) - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime(*): 10ms (10371683) - TotalRawHdfsReadTime(*): 177ms (177980078) - TotalReadThroughput: 219.7 KiB/s (224930) - TotalTime: 51ms (51408037) Instance 244b213cd61a74ca:145f38ad00000003 (host=chia4.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008361204013378) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 76.7 MiB (80403105) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 76.7 MiB (80403105) - RowsProduced: 5,120 (5120) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1.60us (1603) - TotalStorageWaitTime: 16ms (16173330) - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 597ms (597035868) - TotalThreadsSysTime: 24ms (24993000) - TotalThreadsUserTime: 555ms (555582000) - TotalThreadsVoluntaryContextSwitches: 4 (4) - TotalTime: 148ms (148649627) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 70ms (70536417) - InactiveTotalTime: 0ns (0) - OpenTime: 118ms (118849209) - ExecTreeOpenTime: 38.79us (38794) - PrepareTime: 29ms (29754318) - ExecTreePrepareTime: 118.96us (118961) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.5 MiB (1612269) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 654.8 MiB/s (686648407) - OverallThroughput: 68.0 MiB/s (71309559) - PeakMemoryUsage: 192 B (192) - RowsReturned: 4,096 (4096) - SerializeBatchTime: 20ms (20572438) - TotalTime: 22ms (22609437) - TransmitDataRPCTime: 2ms (2348027) - UncompressedRowBatchSize: 4.5 MiB (4697718) CodeGen - CodegenTime: 5ms (5372644) - CompileTime: 34ms (34237955) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 83ms (83477691) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 26ms (26487380) - TotalTime: 145ms (145096645) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (8.368200836820083E-4) - AverageScannerThreadConcurrency: 0.00 (8.368200836820083E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 75.0 MiB (78663680) - PerReadThreadRawHdfsThroughput: 701.8 MiB/s (735894925) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 477,610 (477610) - RowsReturned: 5,120 (5120) - RowsReturnedRate: 72278 per second (72278) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 478ms (478195625) - DelimiterParseTime: 396ms (396344050) - MaterializeTupleTime(*): 63ms (63787137) - ScannerThreadsSysTime: 22ms (22996000) - ScannerThreadsUserTime: 438ms (438937000) - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime(*): 4ms (4870836) - TotalRawHdfsReadTime(*): 182ms (182476138) - TotalReadThroughput: 219.3 KiB/s (224553) - TotalTime: 70ms (70837234) Instance 244b213cd61a74ca:145f38ad00000002 (host=chia3.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008347245409015) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 76.7 MiB (80403105) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 76.7 MiB (80403105) - RowsProduced: 5,120 (5120) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 2.29us (2290) - TotalStorageWaitTime: 19ms (19244826) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 592ms (592503751) - TotalThreadsSysTime: 30ms (30033000) - TotalThreadsUserTime: 542ms (542944000) - TotalThreadsVoluntaryContextSwitches: 4 (4) - TotalTime: 147ms (147583589) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 61ms (61244314) - InactiveTotalTime: 0ns (0) - OpenTime: 118ms (118143808) - ExecTreeOpenTime: 39.81us (39806) - PrepareTime: 29ms (29387938) - ExecTreePrepareTime: 115.76us (115757) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.5 MiB (1603904) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 593.2 MiB/s (622050368) - OverallThroughput: 46.9 MiB/s (49174954) - PeakMemoryUsage: 192 B (192) - RowsReturned: 4,096 (4096) - SerializeBatchTime: 30ms (30493432) - TotalTime: 32ms (32616278) - TransmitDataRPCTime: 2ms (2578415) - UncompressedRowBatchSize: 4.5 MiB (4697499) CodeGen - CodegenTime: 5ms (5137292) - CompileTime: 34ms (34801621) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 82ms (82318656) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 26ms (26117017) - TotalTime: 144ms (144121684) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (8.35421888053467E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 75.0 MiB (78663680) - PerReadThreadRawHdfsThroughput: 779.1 MiB/s (816897094) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 478,515 (478515) - RowsReturned: 5,120 (5120) - RowsReturnedRate: 83205 per second (83205) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 474ms (474369436) - DelimiterParseTime: 389ms (389893146) - MaterializeTupleTime(*): 63ms (63356839) - ScannerThreadsSysTime: 30ms (30007000) - ScannerThreadsUserTime: 425ms (425101000) - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime(*): 4ms (4668340) - TotalRawHdfsReadTime(*): 164ms (164382105) - TotalReadThroughput: 218.9 KiB/s (224179) - TotalTime: 61ms (61534699) Instance 244b213cd61a74ca:145f38ad00000005 (host=chia5.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008375209380234) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 76.7 MiB (80403105) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 76.7 MiB (80403105) - RowsProduced: 5,120 (5120) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 2.32us (2323) - TotalStorageWaitTime: 13ms (13390155) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 603ms (603592812) - TotalThreadsSysTime: 15ms (15998000) - TotalThreadsUserTime: 573ms (573982000) - TotalThreadsVoluntaryContextSwitches: 4 (4) - TotalTime: 147ms (147321116) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 61ms (61610626) - InactiveTotalTime: 0ns (0) - OpenTime: 117ms (117710170) - ExecTreeOpenTime: 39.18us (39183) - PrepareTime: 29ms (29553887) - ExecTreePrepareTime: 123.37us (123367) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.5 MiB (1616249) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 606.1 MiB/s (635575581) - OverallThroughput: 49.4 MiB/s (51784153) - PeakMemoryUsage: 192 B (192) - RowsReturned: 4,096 (4096) - SerializeBatchTime: 29ms (29096599) - TotalTime: 31ms (31211266) - TransmitDataRPCTime: 2ms (2542969) - UncompressedRowBatchSize: 4.5 MiB (4701222) CodeGen - CodegenTime: 5ms (5333889) - CompileTime: 34ms (34189732) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 82ms (82420645) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 26ms (26250190) - TotalTime: 143ms (143701977) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (8.375209380234506E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 75.0 MiB (78663680) - PerReadThreadRawHdfsThroughput: 853.4 MiB/s (894896465) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 477,502 (477502) - RowsReturned: 5,120 (5120) - RowsReturnedRate: 82687 per second (82687) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 485ms (485891692) - DelimiterParseTime: 406ms (406881356) - MaterializeTupleTime(*): 63ms (63842164) - ScannerThreadsSysTime: 14ms (14983000) - ScannerThreadsUserTime: 457ms (457493000) - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsOpenFileTime(*): 4ms (4405715) - TotalRawHdfsReadTime(*): 150ms (150054525) - TotalReadThroughput: 219.7 KiB/s (224930) - TotalTime: 61ms (61919542) Instance 244b213cd61a74ca:145f38ad00000001 (host=chia2.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/33.28 MB - AverageThreadTokens: 1.00 (1.0008361204013378) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 35.3 MiB (37036705) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 35.3 MiB (37036705) - RowsProduced: 4,096 (4096) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1.65us (1647) - TotalStorageWaitTime: 13ms (13627405) - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 268ms (268659519) - TotalThreadsSysTime: 8ms (8978000) - TotalThreadsUserTime: 245ms (245834000) - TotalThreadsVoluntaryContextSwitches: 1 (1) - TotalTime: 147ms (147141561) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 58ms (58437403) - InactiveTotalTime: 0ns (0) - OpenTime: 117ms (117944720) - ExecTreeOpenTime: 36.59us (36593) - PrepareTime: 29ms (29143302) - ExecTreePrepareTime: 118.86us (118857) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.2 MiB (1293846) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 595.3 MiB/s (624263785) - OverallThroughput: 46.4 MiB/s (48674690) - PeakMemoryUsage: 192 B (192) - RowsReturned: 3,072 (3072) - SerializeBatchTime: 24ms (24946570) - TotalTime: 26ms (26581494) - TransmitDataRPCTime: 2ms (2072595) - UncompressedRowBatchSize: 3.6 MiB (3763632) CodeGen - CodegenTime: 5ms (5154977) - CompileTime: 34ms (34271466) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 82ms (82562349) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 25ms (25975327) - TotalTime: 143ms (143659559) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/33.28 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (8.361204013377926E-4) - BytesRead: 33.3 MiB (34896887) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.7 MiB (35297280) - PerReadThreadRawHdfsThroughput: 904.4 MiB/s (948320509) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 124,897 (124897) - RowsReturned: 4,096 (4096) - RowsReturnedRate: 69737 per second (69737) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 150ms (150724110) - DelimiterParseTime: 117ms (117584449) - MaterializeTupleTime(*): 18ms (18271713) - ScannerThreadsSysTime: 7ms (7947000) - ScannerThreadsUserTime: 129ms (129152000) - ScannerThreadsVoluntaryContextSwitches: 1 (1) - TotalRawHdfsOpenFileTime(*): 5ms (5529754) - TotalRawHdfsReadTime(*): 36ms (36798621) - TotalReadThroughput: 57.0 KiB/s (58355) - TotalTime: 58ms (58734242)