Query (id=f64da6a899944136:22dcac6800000000) Summary Session ID: 45422724aac24718:12f97cc20cff7eb0 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2019-08-29 11:04:19.225105000 End Time: 2019-08-29 12:14:38.407351000 Query Type: QUERY Query State: EXCEPTION Query Status: Query f64da6a899944136:22dcac6800000000 expired due to client inactivity (timeout is 10m) Impala Version: impalad version 2.11.0-cdh5.14.4 RELEASE (build 20e635646a13347800fad36a7d0b1da25ab32404) User: hue/chia.haas.berkeley.edu@CAMPUS.BERKELEY.EDU Connected User: hue/chia.haas.berkeley.edu@CAMPUS.BERKELEY.EDU Delegated User: Network Address: ::ffff:10.0.0.10:52714 Default Db: derived_v4 Sql Statement: select * from external_data.physician_compare where primary_specialty = 'INTERNAL MEDICINE' Coordinator: chia.haas.berkeley.edu:22000 Query Options (set by configuration): QUERY_TIMEOUT_S=600 Query Options (set by configuration and planner): QUERY_TIMEOUT_S=600,MT_DOP=0 Plan: ---------------- Max Per-Host Resource Reservation: Memory=0B Per-Host Resource Estimates: Memory=176.00MB WARNING: The following tables are missing relevant table and/or column statistics. external_data.physician_compare F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=0B mem-reservation=0B PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 01:EXCHANGE [UNPARTITIONED] | mem-estimate=0B mem-reservation=0B | tuple-ids=0 row-size=615B cardinality=unavailable | F00:PLAN FRAGMENT [RANDOM] hosts=5 instances=5 Per-Host Resources: mem-estimate=176.00MB mem-reservation=0B 00:SCAN HDFS [external_data.physician_compare, RANDOM] partitions=1/1 files=1 size=545.28MB predicates: primary_specialty = 'INTERNAL MEDICINE' stats-rows=unavailable extrapolated-rows=disabled table stats: rows=unavailable size=0B column stats: unavailable parquet dictionary predicates: primary_specialty = 'INTERNAL MEDICINE' mem-estimate=176.00MB mem-reservation=0B tuple-ids=0 row-size=615B cardinality=unavailable ---------------- Estimated Per-Host Mem: 184549376 Tables Missing Stats: external_data.physician_compare Per Host Min Reservation: chia.haas.berkeley.edu:22000(0) chia1.haas.berkeley.edu:22000(0) chia2.haas.berkeley.edu:22000(0) chia3.haas.berkeley.edu:22000(0) chia4.haas.berkeley.edu:22000(0) chia5.haas.berkeley.edu:22000(0) Request Pool: root.hue Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail -------------------------------------------------------------------------------------------------------------------------- 01:EXCHANGE 1 189.890ms 189.890ms 1.02K -1 0 0 UNPARTITIONED 00:SCAN HDFS 5 98.859ms 104.753ms 22.53K -1 78.33 MB 176.00 MB external_data.physician_compare Errors: - ComputeScanRangeAssignmentTimer: 66.33us (66334) - InactiveTotalTime: 0ns (0) - TotalTime: 0ns (0) Planner Timeline Analysis finished: 2ms (2060686) Value transfer graph computed: 2ms (2246150) Single node plan created: 4ms (4579362) Runtime filters computed: 4ms (4719677) Distributed plan created: 4ms (4778560) Lineage info computed: 5ms (5153412) Planning finished: 7ms (7606608) Query Timeline Query submitted: 134.63us (134631) Planning finished: 9ms (9863116) Submit for admission: 10ms (10932057) Completed admission: 11ms (11024763) Ready to start on 6 backends: 11ms (11523684) All 6 execution backends (6 fragment instances) started: 13ms (13912378) Rows available: 222ms (222456183) First row fetched: 1.45s (1453264453) Cancelled: 10.0m (602357748384) Unregister query: 1.2h (4219182267469) ImpalaServer - ClientFetchWaitTimer: 1.2h (4218958351201) - InactiveTotalTime: 0ns (0) - RowMaterializationTimer: 1ms (1152331) - TotalTime: 0ns (0) Execution Profile f64da6a899944136:22dcac6800000000 Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- Backend startup latencies: Count: 6, min / max: 1ms / 2ms, 25th %-ile: 1ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms Per Node Peak Memory Usage: chia.haas.berkeley.edu:22000(10.79 MB) chia1.haas.berkeley.edu:22000(79.99 MB) chia2.haas.berkeley.edu:22000(35.32 MB) chia3.haas.berkeley.edu:22000(75.02 MB) chia4.haas.berkeley.edu:22000(76.68 MB) chia5.haas.berkeley.edu:22000(78.33 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0ns (0) - InactiveTotalTime: 0ns (0) - NumBackends: 6 (6) - NumFragmentInstances: 6 (6) - NumFragments: 2 (2) - TotalTime: 212ms (212007563) Averaged Fragment F01 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:10m2s max:10m2s mean: 10m2s stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 10.8 MiB (11284344) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 10.8 MiB (11309624) - RowsProduced: 1,024 (1024) - TotalNetworkReceiveTime: 189ms (189851796) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 189ms (189874668) - TotalThreadsSysTime: 2.00us (2000) - TotalThreadsUserTime: 22.00us (22000) - TotalThreadsVoluntaryContextSwitches: 1 (1) - TotalTime: 209ms (209126809) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 38.88us (38883) - InactiveTotalTime: 0ns (0) - OpenTime: 189ms (189878042) - ExecTreeOpenTime: 189ms (189853920) - PrepareTime: 19ms (19198916) - ExecTreePrepareTime: 69.14us (69143) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - TotalTime: 0ns (0) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 18ms (18124391) - TotalTime: 18ms (18791990) EXCHANGE_NODE (id=1) - ConvertRowBatchTime: 37.08us (37080) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1,024 (1024) - RowsReturnedRate: 5392 per second (5392) - TotalTime: 189ms (189890144) DataStreamReceiver - BytesReceived: 5.2 MiB (5477652) - DeserializeRowBatchTimer: 16ms (16880962) - FirstBatchArrivalWaitTime: 189ms (189845344) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 10.8 MiB (11276152) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Coordinator Fragment F01 Instance f64da6a899944136:22dcac6800000000 (host=chia.haas.berkeley.edu:22000) - AverageThreadTokens: 0.00 (0.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 10.8 MiB (11284344) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 10.8 MiB (11309624) - RowsProduced: 1,024 (1024) - TotalNetworkReceiveTime: 189ms (189851796) - TotalNetworkSendTime: 0ns (0) - TotalStorageWaitTime: 0ns (0) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 189ms (189874668) - TotalThreadsSysTime: 2.00us (2000) - TotalThreadsUserTime: 22.00us (22000) - TotalThreadsVoluntaryContextSwitches: 1 (1) - TotalTime: 209ms (209126809) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 38.88us (38883) - InactiveTotalTime: 0ns (0) - OpenTime: 189ms (189878042) - ExecTreeOpenTime: 189ms (189853920) - PrepareTime: 19ms (19198916) - ExecTreePrepareTime: 69.14us (69143) - TotalTime: 0ns (0) PLAN_ROOT_SINK - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - TotalTime: 0ns (0) CodeGen - CodegenTime: 0ns (0) - CompileTime: 0ns (0) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - PrepareTime: 18ms (18124391) - TotalTime: 18ms (18791990) EXCHANGE_NODE (id=1) - ConvertRowBatchTime: 37.08us (37080) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 0 B (0) - RowsReturned: 1,024 (1024) - RowsReturnedRate: 5392 per second (5392) - TotalTime: 189ms (189890144) DataStreamReceiver - BytesReceived: 5.2 MiB (5477652) - DeserializeRowBatchTimer: 16ms (16880962) - FirstBatchArrivalWaitTime: 189ms (189845344) - InactiveTotalTime: 189ms (189846209) - PeakMemoryUsage: 10.8 MiB (11276152) - SendersBlockedTimer: 0ns (0) - SendersBlockedTotalTimer(*): 0ns (0) - TotalTime: 0ns (0) Averaged Fragment F00 split sizes: min: 33.28 MB, max: 128.00 MB, avg: 109.06 MB, stddev: 37.89 MB completion times: min:10m2s max:10m2s mean: 10m2s stddev:304.483us execution rates: min:56.58 KB/sec max:217.60 KB/sec mean:185.40 KB/sec stddev:64.41 KB/sec num instances: 5 - AverageThreadTokens: 1.00 (1.000666394902069) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 69.1 MiB (72423687) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 69.1 MiB (72423687) - RowsProduced: 4,505 (4505) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1.92us (1920) - TotalStorageWaitTime: 63ms (63989566) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 588ms (588583469) - TotalThreadsSysTime: 26ms (26556000) - TotalThreadsUserTime: 497ms (497371400) - TotalThreadsVoluntaryContextSwitches: 3 (3) - TotalTime: 151ms (151744473) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 98ms (98262492) - InactiveTotalTime: 0ns (0) - OpenTime: 119ms (119557029) - ExecTreeOpenTime: 48.52us (48525) - PrepareTime: 32ms (32099907) - ExecTreePrepareTime: 244.74us (244742) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.4 MiB (1418632) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 581.8 MiB/s (610063729) - OverallThroughput: 52.2 MiB/s (54733726) - PeakMemoryUsage: 192 B (192) - RowsReturned: 3,481 (3481) - SerializeBatchTime: 25ms (25207290) - TotalTime: 27ms (27122406) - TransmitDataRPCTime: 2ms (2359864) - UncompressedRowBatchSize: 3.9 MiB (4138719) CodeGen - CodegenTime: 5ms (5926757) - CompileTime: 35ms (35813633) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 82ms (82626382) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 28ms (28193634) - TotalTime: 147ms (147538081) HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (6.663949020691215E-4) - BytesRead: 109.1 MiB (114405988) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 67.4 MiB (70684262) - PerReadThreadRawHdfsThroughput: 769.9 MiB/s (807313673) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 407,271 (407271) - RowsReturned: 4,505 (4505) - RowsReturnedRate: 46208 per second (46208) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 469ms (469036520) - DelimiterParseTime: 348ms (348518144) - MaterializeTupleTime(*): 54ms (54694885) - ScannerThreadsSysTime: 24ms (24579800) - ScannerThreadsUserTime: 380ms (380319200) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 39ms (39150071) - TotalRawHdfsReadTime(*): 143ms (143588962) - TotalReadThroughput: 186.2 KiB/s (190641) - TotalTime: 98ms (98859502) Fragment F00 Instance f64da6a899944136:22dcac6800000001 (host=chia2.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/33.28 MB - AverageThreadTokens: 1.00 (1.0) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 35.3 MiB (37036705) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 35.3 MiB (37036705) - RowsProduced: 4,096 (4096) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1.77us (1771) - TotalStorageWaitTime: 52ms (52221540) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 311ms (311114260) - TotalThreadsSysTime: 9ms (9934000) - TotalThreadsUserTime: 247ms (247352000) - TotalThreadsVoluntaryContextSwitches: 2 (2) - TotalTime: 153ms (153673275) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 98ms (98548349) - InactiveTotalTime: 0ns (0) - OpenTime: 122ms (122121097) - ExecTreeOpenTime: 59.68us (59677) - PrepareTime: 31ms (31484439) - ExecTreePrepareTime: 170.76us (170760) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.2 MiB (1293846) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 541.0 MiB/s (567250659) - OverallThroughput: 44.6 MiB/s (46725506) - PeakMemoryUsage: 192 B (192) - RowsReturned: 3,072 (3072) - SerializeBatchTime: 25ms (25874269) - TotalTime: 27ms (27690358) - TransmitDataRPCTime: 2ms (2280907) - UncompressedRowBatchSize: 3.6 MiB (3763632) CodeGen - CodegenTime: 5ms (5708947) - CompileTime: 38ms (38981608) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 81ms (81997191) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 27ms (27852400) - TotalTime: 149ms (149735510) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/33.28 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (0.0) - BytesRead: 33.3 MiB (34896887) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.7 MiB (35297280) - PerReadThreadRawHdfsThroughput: 823.2 MiB/s (863186937) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 124,897 (124897) - RowsReturned: 4,096 (4096) - RowsReturnedRate: 41378 per second (41378) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 189ms (189004217) - DelimiterParseTime: 116ms (116615110) - MaterializeTupleTime(*): 18ms (18874731) - ScannerThreadsSysTime: 7ms (7938000) - ScannerThreadsUserTime: 128ms (128002000) - ScannerThreadsVoluntaryContextSwitches: 2 (2) - TotalRawHdfsOpenFileTime(*): 40ms (40450588) - TotalRawHdfsReadTime(*): 40ms (40427960) - TotalReadThroughput: 57.0 KiB/s (58355) - TotalTime: 98ms (98987629) Instance f64da6a899944136:22dcac6800000005 (host=chia5.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008305647840532) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 78.3 MiB (82137761) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 78.3 MiB (82137761) - RowsProduced: 4,096 (4096) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 1.89us (1890) - TotalStorageWaitTime: 46ms (46422251) - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 671ms (671061393) - TotalThreadsSysTime: 27ms (27917000) - TotalThreadsUserTime: 596ms (596160000) - TotalThreadsVoluntaryContextSwitches: 2 (2) - TotalTime: 153ms (153324556) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 99ms (99318030) - InactiveTotalTime: 0ns (0) - OpenTime: 118ms (118931096) - ExecTreeOpenTime: 54.88us (54884) - PrepareTime: 34ms (34277794) - ExecTreePrepareTime: 343.14us (343137) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.2 MiB (1291355) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 569.8 MiB/s (597436033) - OverallThroughput: 45.1 MiB/s (47278454) - PeakMemoryUsage: 192 B (192) - RowsReturned: 3,072 (3072) - SerializeBatchTime: 25ms (25633627) - TotalTime: 27ms (27313816) - TransmitDataRPCTime: 2ms (2161495) - UncompressedRowBatchSize: 3.6 MiB (3761032) CodeGen - CodegenTime: 6ms (6443216) - CompileTime: 35ms (35402562) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 82ms (82439346) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 29ms (29805868) - TotalTime: 148ms (148574271) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (8.305647840531562E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 76.7 MiB (80398336) - PerReadThreadRawHdfsThroughput: 750.8 MiB/s (787300354) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 477,502 (477502) - RowsReturned: 4,096 (4096) - RowsReturnedRate: 40883 per second (40883) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 552ms (552139130) - DelimiterParseTime: 438ms (438620725) - MaterializeTupleTime(*): 65ms (65036820) - ScannerThreadsSysTime: 26ms (26995000) - ScannerThreadsUserTime: 478ms (478921000) - ScannerThreadsVoluntaryContextSwitches: 2 (2) - TotalRawHdfsOpenFileTime(*): 34ms (34470177) - TotalRawHdfsReadTime(*): 170ms (170561671) - TotalReadThroughput: 217.8 KiB/s (223061) - TotalTime: 100ms (100188195) Instance f64da6a899944136:22dcac6800000004 (host=chia1.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008305647840532) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 80.0 MiB (83872417) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 80.0 MiB (83872417) - RowsProduced: 3,072 (3072) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 850ns (850) - TotalStorageWaitTime: 109ms (109983495) - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 671ms (671019576) - TotalThreadsSysTime: 33ms (33933000) - TotalThreadsUserTime: 526ms (526739000) - TotalThreadsVoluntaryContextSwitches: 3 (3) - TotalTime: 153ms (153064734) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 104ms (104164005) - InactiveTotalTime: 0ns (0) - OpenTime: 119ms (119658547) - ExecTreeOpenTime: 43.28us (43284) - PrepareTime: 33ms (33298698) - ExecTreePrepareTime: 237.28us (237276) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 943.2 KiB (965808) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 637.4 MiB/s (668344549) - OverallThroughput: 71.8 MiB/s (75319222) - PeakMemoryUsage: 192 B (192) - RowsReturned: 2,048 (2048) - SerializeBatchTime: 11ms (11891353) - TotalTime: 12ms (12822862) - TransmitDataRPCTime: 1ms (1445075) - UncompressedRowBatchSize: 2.7 MiB (2828308) CodeGen - CodegenTime: 6ms (6227024) - CompileTime: 35ms (35513731) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 83ms (83036738) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 29ms (29115753) - TotalTime: 148ms (148592820) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (8.305647840531562E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 78.3 MiB (82132992) - PerReadThreadRawHdfsThroughput: 728.9 MiB/s (764304357) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 477,833 (477833) - RowsReturned: 3,072 (3072) - RowsReturnedRate: 29326 per second (29326) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 551ms (551370270) - DelimiterParseTime: 378ms (378681380) - MaterializeTupleTime(*): 60ms (60704475) - ScannerThreadsSysTime: 32ms (32948000) - ScannerThreadsUserTime: 408ms (408366000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 62ms (62612181) - TotalRawHdfsReadTime(*): 175ms (175693443) - TotalReadThroughput: 217.8 KiB/s (223061) - TotalTime: 104ms (104753120) Instance f64da6a899944136:22dcac6800000002 (host=chia3.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008347245409015) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 75.0 MiB (78668449) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 75.0 MiB (78668449) - RowsProduced: 6,144 (6144) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 2.74us (2738) - TotalStorageWaitTime: 58ms (58723818) - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 638ms (638705608) - TotalThreadsSysTime: 30ms (30992000) - TotalThreadsUserTime: 548ms (548644000) - TotalThreadsVoluntaryContextSwitches: 6 (6) - TotalTime: 149ms (149577694) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 88ms (88590488) - InactiveTotalTime: 0ns (0) - OpenTime: 118ms (118812969) - ExecTreeOpenTime: 44.23us (44233) - PrepareTime: 30ms (30698052) - ExecTreePrepareTime: 325.53us (325528) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.8 MiB (1929886) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 682.9 MiB/s (716081696) - OverallThroughput: 53.0 MiB/s (55591645) - PeakMemoryUsage: 192 B (192) - RowsReturned: 5,120 (5120) - SerializeBatchTime: 32ms (32347978) - TotalTime: 34ms (34715396) - TransmitDataRPCTime: 2ms (2695064) - UncompressedRowBatchSize: 5.4 MiB (5642907) CodeGen - CodegenTime: 5ms (5466346) - CompileTime: 34ms (34844708) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 82ms (82874786) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 27ms (27030701) - TotalTime: 145ms (145634430) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (8.347245409015025E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 73.4 MiB (76929024) - PerReadThreadRawHdfsThroughput: 762.8 MiB/s (799851631) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 478,515 (478515) - RowsReturned: 6,144 (6144) - RowsReturnedRate: 68791 per second (68791) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 1 (1) - ScannerThreadsTotalWallClockTime: 519ms (519903396) - DelimiterParseTime: 395ms (395710895) - MaterializeTupleTime(*): 63ms (63512529) - ScannerThreadsSysTime: 28ms (28010000) - ScannerThreadsUserTime: 433ms (433166000) - ScannerThreadsVoluntaryContextSwitches: 6 (6) - TotalRawHdfsOpenFileTime(*): 19ms (19622862) - TotalRawHdfsReadTime(*): 167ms (167885216) - TotalReadThroughput: 218.9 KiB/s (224179) - TotalTime: 89ms (89312982) Instance f64da6a899944136:22dcac6800000003 (host=chia4.haas.berkeley.edu:22000) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AverageThreadTokens: 1.00 (1.0008361204013378) - BloomFilterBytes: 0 B (0) - InactiveTotalTime: 0ns (0) - PeakMemoryUsage: 76.7 MiB (80403105) - PeakReservation: 0 B (0) - PeakUsedReservation: 0 B (0) - PerHostPeakMemUsage: 76.7 MiB (80403105) - RowsProduced: 5,120 (5120) - TotalNetworkReceiveTime: 0ns (0) - TotalNetworkSendTime: 2.35us (2351) - TotalStorageWaitTime: 52ms (52596730) - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 651ms (651016512) - TotalThreadsSysTime: 30ms (30004000) - TotalThreadsUserTime: 567ms (567962000) - TotalThreadsVoluntaryContextSwitches: 3 (3) - TotalTime: 149ms (149082109) Fragment Instance Lifecycle Timings - ExecTime: 0ns (0) - ExecTreeExecTime: 100ms (100691588) - InactiveTotalTime: 0ns (0) - OpenTime: 118ms (118261438) - ExecTreeOpenTime: 40.55us (40549) - PrepareTime: 30ms (30740552) - ExecTreePrepareTime: 147.01us (147009) - TotalTime: 0ns (0) DataStreamSender (dst_id=1) - BytesSent: 1.5 MiB (1612269) - InactiveTotalTime: 0ns (0) - NetworkThroughput(*): 478.0 MiB/s (501205708) - OverallThroughput: 46.5 MiB/s (48753807) - PeakMemoryUsage: 192 B (192) - RowsReturned: 4,096 (4096) - SerializeBatchTime: 30ms (30289226) - TotalTime: 33ms (33069602) - TransmitDataRPCTime: 3ms (3216781) - UncompressedRowBatchSize: 4.5 MiB (4697718) CodeGen - CodegenTime: 5ms (5788253) - CompileTime: 34ms (34325559) - InactiveTotalTime: 0ns (0) - LoadTime: 0ns (0) - ModuleBitcodeSize: 1.9 MiB (1955788) - NumFunctions: 72 (72) - NumInstructions: 1,511 (1511) - OptimizationTime: 82ms (82783849) - PeakMemoryUsage: 755.5 KiB (773632) - PrepareTime: 27ms (27163451) - TotalTime: 145ms (145153376) HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled - AverageHdfsReadThreadConcurrency: 0.00 (0.0) - AverageScannerThreadConcurrency: 0.00 (8.361204013377926E-4) - BytesRead: 128.1 MiB (134283264) - BytesReadDataNodeCache: 0 B (0) - BytesReadLocal: 0 B (0) - BytesReadRemoteUnexpected: 0 B (0) - BytesReadShortCircuit: 0 B (0) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0ns (0) - InactiveTotalTime: 0ns (0) - MaxCompressedTextFileLength: 0 B (0) - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 75.0 MiB (78663680) - PerReadThreadRawHdfsThroughput: 783.8 MiB/s (821925088) - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0ns (0) - RowBatchQueuePutWaitTime: 0ns (0) - RowsRead: 477,610 (477610) - RowsReturned: 5,120 (5120) - RowsReturnedRate: 50665 per second (50665) - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 532ms (532765590) - DelimiterParseTime: 412ms (412962612) - MaterializeTupleTime(*): 65ms (65345872) - ScannerThreadsSysTime: 27ms (27008000) - ScannerThreadsUserTime: 453ms (453141000) - ScannerThreadsVoluntaryContextSwitches: 3 (3) - TotalRawHdfsOpenFileTime(*): 38ms (38594548) - TotalRawHdfsReadTime(*): 163ms (163376524) - TotalReadThroughput: 219.3 KiB/s (224553) - TotalTime: 101ms (101055588)