Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

impala submission control issue

impala submission control issue

Super Collaborator

Hello All,

Will appreciate any insight of what i'm missing in my configuration to get the impala submission control working as expected ....

a- Here is my placement Placement Rule:
1- Use the pool Specified at run time., only if the pool exists.
2- Use the pool root.[secondary group].
3- Use the pool root.default.

b- Here is the impala resource pools

Name Max Memory Max Running Queries Max Queued Queries Queue Timeout Default Query Memory Limit Access Control
root.impiaas Unlimited Unlimited 200 1 minute(s) No Default Custom
root.impreports Unlimited Unlimited 200 1 minute(s) No Default Custom


C- Here is my Access control for root.impiaas pool:

Allow these users and groups to submit to this pool

Users: fawzea

The same for impreports the allowed user and groups to submit to pool is only nira.


LDAP groups for fawzea user:

Member in groups:

- dba
- impiaas
- mysql

LDAP group for nira user
Member in groups:
- dba
- impreports
- mysql

 

Impala Queries UI shows:

1- select count(*) from dv_entersiteevent_prq_remote where year=2017

User: fawzea
Pool: root.impiaas

2- select count(*) from dv_entersiteevent_prq_remote where year=2017 and month=10
User: nira
Pool: root.impreports

As shown above that the queries submitted based on the secondary group rule.

When i'm running the following with fawzea user:

set request_pool=impreports;
select count(*) from dv_entersiteevent_prq_remote where year=2017 and month=10;

I can see that the query is running with impreports pool.


What i'm missing here in my configuration to get this works as expected.
==================================================================================

The config:

[root@aopr-dhc001 ~]# cat /var/run/cloudera-scm-agent/process/4558-impala-IMPALAD/impala-conf/fair-scheduler.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<allocations>
<queue name="root">
<weight>1.0</weight>
<schedulingPolicy>fair</schedulingPolicy>
<queue name="impiaas">
<weight>1.0</weight>
<schedulingPolicy>fair</schedulingPolicy>
<aclSubmitApps>fawzea </aclSubmitApps>
</queue>
<queue name="impreports">
<weight>1.0</weight>
<schedulingPolicy>fair</schedulingPolicy>
<aclSubmitApps>nira </aclSubmitApps>
</queue>
</queue>
<defaultQueueSchedulingPolicy>fair</defaultQueueSchedulingPolicy>
<queuePlacementPolicy>
<rule name="specified" create="false"/>
<rule name="secondaryGroupExistingQueue"/>
<rule name="default"/>
</queuePlacementPolicy>
</allocations>

==============================================

Query profile and log:

Query Runtime Profile:
Query (id=c843dae48ce9b289:6ad3ffc000000000):
Summary:
Session ID: ca420ac477772b56:51ae460ce9e44999
Session Type: BEESWAX
Start Time: 2018-02-08 01:02:01.023558000
End Time: 2018-02-08 01:02:01.501506000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.11.0-cdh5.14.0 RELEASE (build d68206561bce6b26762d62c01a78e6cd27aa7690)
User: fawzea
Connected User: fawzea
Delegated User:
Network Address: ::ffff:10.16.144.131:50078
Default Db: default
Sql Statement: select count(*) from dv_entersiteevent_prq_remote where year=2017 and month=10
Coordinator: aopr-dhc001.lpdomain.com:22000
Query Options (set by configuration): REQUEST_POOL=impreports
Query Options (set by configuration and planner): REQUEST_POOL=impreports,MT_DOP=0
Plan:
----------------
Max Per-Host Resource Reservation: Memory=0B
Per-Host Resource Estimates: Memory=21.00MB
WARNING: The following tables are missing relevant table and/or column statistics.
default.dv_entersiteevent_prq_remote

F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B
|
03:AGGREGATE [FINALIZE]
| output: count:merge(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=1 row-size=8B cardinality=1
|
02:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=1 row-size=8B cardinality=1
|
F00:PLAN FRAGMENT [RANDOM] hosts=5 instances=5
Per-Host Resources: mem-estimate=11.00MB mem-reservation=0B
01:AGGREGATE
| output: sum_init_zero(default.dv_entersiteevent_prq_remote.parquet-stats: num_rows)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=1 row-size=8B cardinality=1
|
00:SCAN HDFS [default.dv_entersiteevent_prq_remote, RANDOM]
partitions=31/441 files=2292 size=1.39GB
stats-rows=unavailable extrapolated-rows=disabled
table stats: rows=unavailable size=unavailable
column stats: all
mem-estimate=1.00MB mem-reservation=0B
tuple-ids=0 row-size=8B cardinality=unavailable
----------------
Estimated Per-Host Mem: 22020096
Tables Missing Stats: default.dv_entersiteevent_prq_remote
Per Host Min Reservation: aopr-dhc001.lpdomain.com:22000(0) aopr-dhc002.lpdomain.com:22000(0) aopr-dhc003.lpdomain.com:22000(0) aopr-dhc004.lpdomain.com:22000(0) aopr-dhc005.lpdomain.com:22000(0)
Request Pool: root.impreports
Admission result: Admitted immediately
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
------------------------------------------------------------------------------------------------------------------------------
03:AGGREGATE 1 222.958us 222.958us 1 1 36.00 KB 10.00 MB FINALIZE
02:EXCHANGE 1 357.508ms 357.508ms 5 1 0 0 UNPARTITIONED
01:AGGREGATE 5 1.027ms 1.541ms 5 1 20.00 KB 10.00 MB
00:SCAN HDFS 5 282.797ms 333.715ms 2.29K -1 2.62 MB 1.00 MB default.dv_entersiteevent_prq_remote
Errors:
Planner Timeline: 13.508ms
- Analysis finished: 3.303ms (3.303ms)
- Value transfer graph computed: 3.465ms (162.667us)
- Single node plan created: 10.352ms (6.886ms)
- Runtime filters computed: 10.677ms (325.364us)
- Distributed plan created: 10.733ms (55.854us)
- Lineage info computed: 10.898ms (164.937us)
- Planning finished: 13.508ms (2.610ms)
Query Timeline: 481.950ms
- Query submitted: 79.971us (79.971us)
- Planning finished: 33.947ms (33.867ms)
- Submit for admission: 40.067ms (6.120ms)
- Completed admission: 40.197ms (130.146us)
- Ready to start on 5 backends: 40.905ms (707.852us)
- All 5 execution backends (6 fragment instances) started: 46.417ms (5.512ms)
- Rows available: 424.548ms (378.131ms)
- First row fetched: 473.488ms (48.939ms)
- Unregister query: 477.951ms (4.462ms)
- ComputeScanRangeAssignmentTimer: 3.980ms
ImpalaServer:
- ClientFetchWaitTimer: 51.659ms
- RowMaterializationTimer: 1.726ms
Execution Profile c843dae48ce9b289:6ad3ffc000000000:(Total: 385.029ms, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 0
Filter routing table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------

Backend startup latencies: Count: 5, min / max: 3ms / 4ms, 25th %-ile: 3ms, 50th %-ile: 4ms, 75th %-ile: 4ms, 90th %-ile: 4ms, 95th %-ile: 4ms, 99.9th %-ile: 4ms
Per Node Peak Memory Usage: aopr-dhc001.lpdomain.com:22000(2.68 MB) aopr-dhc002.lpdomain.com:22000(2.54 MB) aopr-dhc003.lpdomain.com:22000(2.54 MB) aopr-dhc004.lpdomain.com:22000(2.41 MB) aopr-dhc005.lpdomain.com:22000(2.56 MB)
- FiltersReceived: 0 (0)
- FinalizationTimer: 0.000ns
- NumBackends: 5 (5)
- NumFragmentInstances: 6 (6)
- NumFragments: 2 (2)
Averaged Fragment F01:(Total: 427.530ms, non-child: 49.428ms, % non-child: 11.56%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:428.693ms max:428.693ms mean: 428.693ms stddev:0.000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 44.02 KB (45072)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.68 MB (2806113)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 357.481ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 406.708ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 7 (7)
Fragment Instance Lifecycle Timings:
- ExecTime: 49.055ms
- ExecTreeExecTime: 4.251us
- OpenTime: 357.663ms
- ExecTreeOpenTime: 357.649ms
- PrepareTime: 20.764ms
- ExecTreePrepareTime: 81.759us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 19.685ms
AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child: 222.958us, % non-child: 0.06%)
- BuildTime: 7.368us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 36.00 KB (36864)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 2.00 /sec
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child: 357.508ms, % non-child: 100.00%)
- ConvertRowBatchTime: 8.881us
- PeakMemoryUsage: 0
- RowsReturned: 5 (5)
- RowsReturnedRate: 13.00 /sec
DataStreamReceiver:
- BytesReceived: 80.00 B (80)
- DeserializeRowBatchTimer: 24.165us
- FirstBatchArrivalWaitTime: 277.194ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Coordinator Fragment F01:
Instance c843dae48ce9b289:6ad3ffc000000000 (host=aopr-dhc001.lpdomain.com:22000):(Total: 427.530ms, non-child: 49.428ms, % non-child: 11.56%)
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 44.02 KB (45072)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.68 MB (2806113)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 357.481ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 406.708ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 7 (7)
Fragment Instance Lifecycle Timings:
- ExecTime: 49.055ms
- ExecTreeExecTime: 4.251us
- OpenTime: 357.663ms
- ExecTreeOpenTime: 357.649ms
- PrepareTime: 20.764ms
- ExecTreePrepareTime: 81.759us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 19.685ms
AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child: 222.958us, % non-child: 0.06%)
- BuildTime: 7.368us
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 36.00 KB (36864)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 2.00 /sec
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child: 357.508ms, % non-child: 100.00%)
- ConvertRowBatchTime: 8.881us
- PeakMemoryUsage: 0
- RowsReturned: 5 (5)
- RowsReturnedRate: 13.00 /sec
DataStreamReceiver:
- BytesReceived: 80.00 B (80)
- DeserializeRowBatchTimer: 24.165us
- FirstBatchArrivalWaitTime: 277.194ms
- PeakMemoryUsage: 4.02 KB (4112)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F00:(Total: 327.360ms, non-child: 1.578ms, % non-child: 0.48%)
split sizes: min: 283.31 MB, max: 284.33 MB, avg: 284.04 MB, stddev: 382.00 KB
completion times: min:299.457ms max:378.950ms mean: 328.881ms stddev:31.507ms
execution rates: min:750.18 MB/sec max:949.01 MB/sec mean:871.27 MB/sec stddev:79.74 MB/sec
num instances: 5
- AverageThreadTokens: 15.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.54 MB (2666852)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.55 MB (2669309)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 303.541us
- TotalStorageWaitTime: 4s821ms
- TotalThreadsInvoluntaryContextSwitches: 104 (104)
- TotalThreadsTotalWallClockTime: 6s835ms
- TotalThreadsSysTime: 31.784ms
- TotalThreadsUserTime: 682.084ms
- TotalThreadsVoluntaryContextSwitches: 2.12K (2121)
Fragment Instance Lifecycle Timings:
- ExecTime: 500.816us
- ExecTreeExecTime: 8.200us
- OpenTime: 302.848ms
- ExecTreeOpenTime: 281.309ms
- PrepareTime: 23.946ms
- ExecTreePrepareTime: 2.513ms
DataStreamSender (dst_id=2):(Total: 269.902us, non-child: 269.902us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 373.13 KB/sec
- OverallThroughput: 63.53 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 14.525us
- TransmitDataRPCTime: 43.122us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 41.687ms, non-child: 41.687ms, % non-child: 100.00%)
- CodegenTime: 1.194ms
- CompileTime: 7.360ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 15 (15)
- NumInstructions: 214 (214)
- OptimizationTime: 13.529ms
- PeakMemoryUsage: 107.00 KB (109568)
- PrepareTime: 20.084ms
AGGREGATION_NODE (id=1):(Total: 283.825ms, non-child: 1.027ms, % non-child: 0.36%)
- BuildTime: 1.259ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 20.00 KB (20480)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 2.00 /sec
- SpilledPartitions: 0 (0)
HDFS_SCAN_NODE (id=0):(Total: 282.797ms, non-child: 282.797ms, % non-child: 100.00%)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 14.40
- BytesRead: 44.52 MB (46686028)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 44.52 MB (46686028)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 44.52 MB (46686028)
- CachedFileHandle**bleep**Count: 0 (0)
- CachedFileHandlesMissCount: 458 (458)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumColumns: 0 (0)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 12 (12)
- NumRowGroups: 458 (458)
- NumScannerThreadsStarted: 24 (24)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 2.50 MB (2625539)
- PerReadThreadRawHdfsThroughput: 1.29 GB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 269.787ms
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 0 (0)
- RowsReturned: 458 (458)
- RowsReturnedRate: 1.64 K/sec
- ScanRangesComplete: 458 (458)
- ScannerThreadsInvoluntaryContextSwitches: 99 (99)
- ScannerThreadsTotalWallClockTime: 6s532ms
- MaterializeTupleTime(*): 70.664us
- ScannerThreadsSysTime: 26.184ms
- ScannerThreadsUserTime: 650.089ms
- ScannerThreadsVoluntaryContextSwitches: 1.67K (1674)
- TotalRawHdfsReadTime(*): 34.416ms
- TotalReadThroughput: 36.86 MB/sec
Fragment F00:
Instance c843dae48ce9b289:6ad3ffc000000004 (host=aopr-dhc004.lpdomain.com:22000):(Total: 377.384ms, non-child: 1.619ms, % non-child: 0.43%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:36/21.95 MB 11:33/19.87 MB 9:27/15.16 MB 5:44/28.07 MB 6:56/33.32 MB 1:38/22.24 MB 10:44/27.90 MB 3:34/20.26 MB 2:45/28.21 MB 7:31/19.33 MB 4:42/26.73 MB 8:34/21.25 MB
MemoryUsage(500.000ms): 1.91 MB
ThreadUsage(500.000ms): 25
- AverageThreadTokens: 25.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.41 MB (2531689)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.41 MB (2531689)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 382.629us
- TotalStorageWaitTime: 5s790ms
- TotalThreadsInvoluntaryContextSwitches: 115 (115)
- TotalThreadsTotalWallClockTime: 8s122ms
- TotalThreadsSysTime: 33.984ms
- TotalThreadsUserTime: 702.879ms
- TotalThreadsVoluntaryContextSwitches: 2.04K (2037)
Fragment Instance Lifecycle Timings:
- ExecTime: 604.005us
- ExecTreeExecTime: 8.338us
- OpenTime: 352.927ms
- ExecTreeOpenTime: 331.921ms
- PrepareTime: 23.783ms
- ExecTreePrepareTime: 2.559ms
DataStreamSender (dst_id=2):(Total: 331.818us, non-child: 331.818us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 289.00 KB/sec
- OverallThroughput: 47.09 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 14.466us
- TransmitDataRPCTime: 54.066us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 40.950ms, non-child: 40.950ms, % non-child: 100.00%)
- CodegenTime: 1.122ms
- CompileTime: 7.318ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 15 (15)
- NumInstructions: 214 (214)
- OptimizationTime: 13.106ms
- PeakMemoryUsage: 107.00 KB (109568)
- PrepareTime: 19.773ms
AGGREGATION_NODE (id=1):(Total: 334.482ms, non-child: 766.527us, % non-child: 0.23%)
ExecOption: Codegen Enabled
- BuildTime: 1.192ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 20.00 KB (20480)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 2.00 /sec
- SpilledPartitions: 0 (0)
HDFS_SCAN_NODE (id=0):(Total: 333.715ms, non-child: 333.715ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:36/21.95 MB 11:33/19.87 MB 9:27/15.16 MB 5:44/28.07 MB 6:56/33.32 MB 1:38/22.24 MB 10:44/27.90 MB 3:34/20.26 MB 2:45/28.21 MB 7:31/19.33 MB 4:42/26.73 MB 8:34/21.25 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 464 out of 464
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
File Formats: PARQUET/Unknown(Skipped):464
BytesRead(500.000ms): 12.27 MB
- FooterProcessingTime: (Avg: 13.555ms ; Min: 5.433ms ; Max: 62.957ms ; Number of samples: 464)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 24.00
- BytesRead: 45.01 MB (47193988)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 45.01 MB (47193988)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 45.01 MB (47193988)
- CachedFileHandle**bleep**Count: 0 (0)
- CachedFileHandlesMissCount: 464 (464)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumColumns: 0 (0)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 12 (12)
- NumRowGroups: 464 (464)
- NumScannerThreadsStarted: 24 (24)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 2.38 MB (2490376)
- PerReadThreadRawHdfsThroughput: 1016.44 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 320.860ms
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 0 (0)
- RowsReturned: 464 (464)
- RowsReturnedRate: 1.39 K/sec
- ScanRangesComplete: 464 (464)
- ScannerThreadsInvoluntaryContextSwitches: 108 (108)
- ScannerThreadsTotalWallClockTime: 7s769ms
- MaterializeTupleTime(*): 75.514us
- ScannerThreadsSysTime: 29.984ms
- ScannerThreadsUserTime: 669.884ms
- ScannerThreadsVoluntaryContextSwitches: 1.60K (1597)
- TotalRawHdfsReadTime(*): 44.279ms
- TotalReadThroughput: 24.53 MB/sec
Instance c843dae48ce9b289:6ad3ffc000000002 (host=aopr-dhc003.lpdomain.com:22000):(Total: 351.014ms, non-child: 1.641ms, % non-child: 0.47%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:35/20.89 MB 7:44/26.64 MB 11:27/17.21 MB 9:32/18.61 MB 5:25/14.56 MB 2:38/24.32 MB 6:31/19.85 MB 8:50/30.76 MB 10:41/25.92 MB 3:31/19.78 MB 4:60/37.59 MB 1:44/27.19 MB
MemoryUsage(500.000ms): 1.16 MB
ThreadUsage(500.000ms): 25
- AverageThreadTokens: 25.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.54 MB (2662753)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.54 MB (2662753)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 342.746us
- TotalStorageWaitTime: 4s796ms
- TotalThreadsInvoluntaryContextSwitches: 93 (93)
- TotalThreadsTotalWallClockTime: 7s286ms
- TotalThreadsSysTime: 27.988ms
- TotalThreadsUserTime: 671.887ms
- TotalThreadsVoluntaryContextSwitches: 2.08K (2079)
Fragment Instance Lifecycle Timings:
- ExecTime: 524.006us
- ExecTreeExecTime: 7.920us
- OpenTime: 327.802ms
- ExecTreeOpenTime: 307.154ms
- PrepareTime: 22.617ms
- ExecTreePrepareTime: 2.650ms
DataStreamSender (dst_id=2):(Total: 348.990us, non-child: 348.990us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 354.47 KB/sec
- OverallThroughput: 44.77 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 15.602us
- TransmitDataRPCTime: 44.080us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 39.215ms, non-child: 39.215ms, % non-child: 100.00%)
- CodegenTime: 1.183ms
- CompileTime: 7.168ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 15 (15)
- NumInstructions: 214 (214)
- OptimizationTime: 12.929ms
- PeakMemoryUsage: 107.00 KB (109568)
- PrepareTime: 18.453ms
AGGREGATION_NODE (id=1):(Total: 309.807ms, non-child: 948.677us, % non-child: 0.31%)
ExecOption: Codegen Enabled
- BuildTime: 1.552ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 20.00 KB (20480)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 3.00 /sec
- SpilledPartitions: 0 (0)
HDFS_SCAN_NODE (id=0):(Total: 308.859ms, non-child: 308.859ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:35/20.89 MB 7:44/26.64 MB 11:27/17.21 MB 9:32/18.61 MB 5:25/14.56 MB 2:38/24.32 MB 6:31/19.85 MB 8:50/30.76 MB 10:41/25.92 MB 3:31/19.78 MB 4:60/37.59 MB 1:44/27.19 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 458 out of 458
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
File Formats: PARQUET/Unknown(Skipped):458
BytesRead(500.000ms): 36.74 MB
- FooterProcessingTime: (Avg: 11.502ms ; Min: 3.668ms ; Max: 23.564ms ; Number of samples: 458)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 24.00
- BytesRead: 44.51 MB (46668432)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 44.51 MB (46668432)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 44.51 MB (46668432)
- CachedFileHandle**bleep**Count: 0 (0)
- CachedFileHandlesMissCount: 458 (458)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumColumns: 0 (0)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 12 (12)
- NumRowGroups: 458 (458)
- NumScannerThreadsStarted: 24 (24)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 2.50 MB (2621440)
- PerReadThreadRawHdfsThroughput: 1.26 GB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 295.595ms
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 0 (0)
- RowsReturned: 458 (458)
- RowsReturnedRate: 1.48 K/sec
- ScanRangesComplete: 458 (458)
- ScannerThreadsInvoluntaryContextSwitches: 91 (91)
- ScannerThreadsTotalWallClockTime: 6s958ms
- MaterializeTupleTime(*): 71.630us
- ScannerThreadsSysTime: 16.989ms
- ScannerThreadsUserTime: 645.891ms
- ScannerThreadsVoluntaryContextSwitches: 1.63K (1630)
- TotalRawHdfsReadTime(*): 34.377ms
- TotalReadThroughput: 73.49 MB/sec
Instance c843dae48ce9b289:6ad3ffc000000001 (host=aopr-dhc005.lpdomain.com:22000):(Total: 308.195ms, non-child: 1.784ms, % non-child: 0.58%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:46/28.97 MB 8:39/24.34 MB 10:33/21.03 MB 3:43/26.50 MB 5:32/20.27 MB 9:32/19.86 MB 6:25/16.01 MB 7:41/24.77 MB 1:42/25.01 MB 0:55/32.51 MB 4:40/24.29 MB 11:33/20.77 MB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.56 MB (2683241)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.56 MB (2683241)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 292.765us
- TotalStorageWaitTime: 4s482ms
- TotalThreadsInvoluntaryContextSwitches: 113 (113)
- TotalThreadsTotalWallClockTime: 6s310ms
- TotalThreadsSysTime: 35.983ms
- TotalThreadsUserTime: 682.884ms
- TotalThreadsVoluntaryContextSwitches: 2.14K (2143)
Fragment Instance Lifecycle Timings:
- ExecTime: 532.572us
- ExecTreeExecTime: 8.377us
- OpenTime: 280.241ms
- ExecTreeOpenTime: 258.652ms
- PrepareTime: 27.342ms
- ExecTreePrepareTime: 2.863ms
DataStreamSender (dst_id=2):(Total: 309.794us, non-child: 309.794us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 356.91 KB/sec
- OverallThroughput: 50.44 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 14.262us
- TransmitDataRPCTime: 43.778us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 44.584ms, non-child: 44.584ms, % non-child: 100.00%)
- CodegenTime: 1.382ms
- CompileTime: 7.346ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 15 (15)
- NumInstructions: 214 (214)
- OptimizationTime: 13.462ms
- PeakMemoryUsage: 107.00 KB (109568)
- PrepareTime: 23.006ms
AGGREGATION_NODE (id=1):(Total: 261.516ms, non-child: 836.656us, % non-child: 0.32%)
ExecOption: Codegen Enabled
- BuildTime: 1.280ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 20.00 KB (20480)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 3.00 /sec
- SpilledPartitions: 0 (0)
HDFS_SCAN_NODE (id=0):(Total: 260.680ms, non-child: 260.680ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:46/28.97 MB 8:39/24.34 MB 10:33/21.03 MB 3:43/26.50 MB 5:32/20.27 MB 9:32/19.86 MB 6:25/16.01 MB 7:41/24.77 MB 1:42/25.01 MB 0:55/32.51 MB 4:40/24.29 MB 11:33/20.77 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 461 out of 461
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
File Formats: PARQUET/Unknown(Skipped):461
- FooterProcessingTime: (Avg: 10.793ms ; Min: 3.087ms ; Max: 20.253ms ; Number of samples: 461)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 0.00
- BytesRead: 44.73 MB (46906407)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 44.73 MB (46906407)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 44.73 MB (46906407)
- CachedFileHandle**bleep**Count: 0 (0)
- CachedFileHandlesMissCount: 461 (461)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumColumns: 0 (0)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 12 (12)
- NumRowGroups: 461 (461)
- NumScannerThreadsStarted: 24 (24)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 2.52 MB (2641928)
- PerReadThreadRawHdfsThroughput: 1.31 GB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 246.691ms
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 0 (0)
- RowsReturned: 461 (461)
- RowsReturnedRate: 1.77 K/sec
- ScanRangesComplete: 461 (461)
- ScannerThreadsInvoluntaryContextSwitches: 109 (109)
- ScannerThreadsTotalWallClockTime: 6s030ms
- MaterializeTupleTime(*): 68.735us
- ScannerThreadsSysTime: 33.983ms
- ScannerThreadsUserTime: 647.889ms
- ScannerThreadsVoluntaryContextSwitches: 1.70K (1696)
- TotalRawHdfsReadTime(*): 33.405ms
- TotalReadThroughput: 0.00 /sec
Instance c843dae48ce9b289:6ad3ffc000000003 (host=aopr-dhc002.lpdomain.com:22000):(Total: 302.116ms, non-child: 1.604ms, % non-child: 0.53%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:27/16.21 MB 11:33/21.23 MB 6:36/23.15 MB 10:41/25.09 MB 1:41/24.13 MB 9:32/18.79 MB 5:41/25.31 MB 2:42/26.49 MB 7:36/22.87 MB 3:51/31.00 MB 4:41/25.99 MB 8:38/23.79 MB
MemoryUsage(500.000ms): 700.35 KB
ThreadUsage(500.000ms): 25
- AverageThreadTokens: 25.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.54 MB (2662753)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.54 MB (2662753)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 300.675us
- TotalStorageWaitTime: 4s661ms
- TotalThreadsInvoluntaryContextSwitches: 90 (90)
- TotalThreadsTotalWallClockTime: 6s256ms
- TotalThreadsSysTime: 30.983ms
- TotalThreadsUserTime: 709.880ms
- TotalThreadsVoluntaryContextSwitches: 2.17K (2175)
Fragment Instance Lifecycle Timings:
- ExecTime: 492.627us
- ExecTreeExecTime: 8.191us
- OpenTime: 277.922ms
- ExecTreeOpenTime: 255.395ms
- PrepareTime: 23.647ms
- ExecTreePrepareTime: 2.243ms
DataStreamSender (dst_id=2):(Total: 194.517us, non-child: 194.517us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 371.86 KB/sec
- OverallThroughput: 80.33 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 14.035us
- TransmitDataRPCTime: 42.019us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 42.676ms, non-child: 42.676ms, % non-child: 100.00%)
- CodegenTime: 1.236ms
- CompileTime: 7.717ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 15 (15)
- NumInstructions: 214 (214)
- OptimizationTime: 14.130ms
- PeakMemoryUsage: 107.00 KB (109568)
- PrepareTime: 20.109ms
AGGREGATION_NODE (id=1):(Total: 257.640ms, non-child: 1.541ms, % non-child: 0.60%)
ExecOption: Codegen Enabled
- BuildTime: 1.252ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 20.00 KB (20480)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 3.00 /sec
- SpilledPartitions: 0 (0)
HDFS_SCAN_NODE (id=0):(Total: 256.099ms, non-child: 256.099ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:27/16.21 MB 11:33/21.23 MB 6:36/23.15 MB 10:41/25.09 MB 1:41/24.13 MB 9:32/18.79 MB 5:41/25.31 MB 2:42/26.49 MB 7:36/22.87 MB 3:51/31.00 MB 4:41/25.99 MB 8:38/23.79 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 459 out of 459
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
File Formats: PARQUET/Unknown(Skipped):459
BytesRead(500.000ms): 43.15 MB
- FooterProcessingTime: (Avg: 11.257ms ; Min: 3.989ms ; Max: 23.004ms ; Number of samples: 459)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 24.00
- BytesRead: 44.52 MB (46678031)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 44.52 MB (46678031)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 44.52 MB (46678031)
- CachedFileHandle**bleep**Count: 0 (0)
- CachedFileHandlesMissCount: 459 (459)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumColumns: 0 (0)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 12 (12)
- NumRowGroups: 459 (459)
- NumScannerThreadsStarted: 24 (24)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 2.50 MB (2621440)
- PerReadThreadRawHdfsThroughput: 1.33 GB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 243.426ms
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 0 (0)
- RowsReturned: 459 (459)
- RowsReturnedRate: 1.79 K/sec
- ScanRangesComplete: 459 (459)
- ScannerThreadsInvoluntaryContextSwitches: 85 (85)
- ScannerThreadsTotalWallClockTime: 5s978ms
- MaterializeTupleTime(*): 67.997us
- ScannerThreadsSysTime: 22.984ms
- ScannerThreadsUserTime: 678.884ms
- ScannerThreadsVoluntaryContextSwitches: 1.72K (1724)
- TotalRawHdfsReadTime(*): 32.626ms
- TotalReadThroughput: 86.30 MB/sec
Instance c843dae48ce9b289:6ad3ffc000000005 (host=aopr-dhc001.lpdomain.com:22000):(Total: 298.093ms, non-child: 1.242ms, % non-child: 0.42%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 9:55/35.38 MB 0:37/24.23 MB 5:37/23.31 MB 11:22/13.79 MB 8:30/18.53 MB 4:33/20.45 MB 1:38/24.06 MB 10:29/18.40 MB 3:39/24.35 MB 2:41/25.45 MB 6:42/27.03 MB 7:47/29.22 MB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.66 MB (2793825)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.68 MB (2806113)
- RowsProduced: 1 (1)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 198.894us
- TotalStorageWaitTime: 4s374ms
- TotalThreadsInvoluntaryContextSwitches: 110 (110)
- TotalThreadsTotalWallClockTime: 6s201ms
- TotalThreadsSysTime: 29.983ms
- TotalThreadsUserTime: 642.892ms
- TotalThreadsVoluntaryContextSwitches: 2.17K (2174)
Fragment Instance Lifecycle Timings:
- ExecTime: 350.873us
- ExecTreeExecTime: 8.173us
- OpenTime: 275.347ms
- ExecTreeOpenTime: 253.425ms
- PrepareTime: 22.341ms
- ExecTreePrepareTime: 2.251ms
DataStreamSender (dst_id=2):(Total: 164.394us, non-child: 164.394us, % non-child: 100.00%)
- BytesSent: 16.00 B (16)
- NetworkThroughput(*): 493.42 KB/sec
- OverallThroughput: 95.05 KB/sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 1 (1)
- SerializeBatchTime: 14.262us
- TransmitDataRPCTime: 31.667us
- UncompressedRowBatchSize: 16.00 B (16)
CodeGen:(Total: 41.008ms, non-child: 41.008ms, % non-child: 100.00%)
- CodegenTime: 1.049ms
- CompileTime: 7.248ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.86 MB (1950880)
- NumFunctions: 15 (15)
- NumInstructions: 214 (214)
- OptimizationTime: 14.018ms
- PeakMemoryUsage: 107.00 KB (109568)
- PrepareTime: 19.080ms
AGGREGATION_NODE (id=1):(Total: 255.678ms, non-child: 1.046ms, % non-child: 0.41%)
ExecOption: Codegen Enabled
- BuildTime: 1.015ms
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 20.00 KB (20480)
- RowsRepartitioned: 0 (0)
- RowsReturned: 1 (1)
- RowsReturnedRate: 3.00 /sec
- SpilledPartitions: 0 (0)
HDFS_SCAN_NODE (id=0):(Total: 254.631ms, non-child: 254.631ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 9:55/35.38 MB 0:37/24.23 MB 5:37/23.31 MB 11:22/13.79 MB 8:30/18.53 MB 4:33/20.45 MB 1:38/24.06 MB 10:29/18.40 MB 3:39/24.35 MB 2:41/25.45 MB 6:42/27.03 MB 7:47/29.22 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 450 out of 450
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
File Formats: PARQUET/Unknown(Skipped):450
- FooterProcessingTime: (Avg: 10.727ms ; Min: 3.813ms ; Max: 23.755ms ; Number of samples: 450)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 0.00
- BytesRead: 43.85 MB (45983284)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 43.85 MB (45983284)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 43.85 MB (45983284)
- CachedFileHandle**bleep**Count: 0 (0)
- CachedFileHandlesMissCount: 450 (450)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumColumns: 0 (0)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 12 (12)
- NumRowGroups: 450 (450)
- NumScannerThreadsStarted: 24 (24)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 2.62 MB (2752512)
- PerReadThreadRawHdfsThroughput: 1.56 GB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 242.362ms
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 0 (0)
- RowsReturned: 450 (450)
- RowsReturnedRate: 1.77 K/sec
- ScanRangesComplete: 450 (450)
- ScannerThreadsInvoluntaryContextSwitches: 106 (106)
- ScannerThreadsTotalWallClockTime: 5s926ms
- MaterializeTupleTime(*): 69.447us
- ScannerThreadsSysTime: 26.984ms
- ScannerThreadsUserTime: 607.897ms
- ScannerThreadsVoluntaryContextSwitches: 1.72K (1723)
- TotalRawHdfsReadTime(*): 27.393ms
- TotalReadThroughput: 0.00 /sec
===================================================

Coordinator log:

01: query (string) = "select count(*) from dv_entersiteevent_prq_remote where year=2017 and month=10",
03: configuration (list) = list<string>[1] {
[0] = "REQUEST_POOL=impreports",
},
04: hadoop_user (string) = "fawzea",
}
I0208 01:07:06.321281 7154 impala-beeswax-server.cc:459] TClientRequest.queryOptions: TQueryOptions {
01: abort_on_error (bool) = false,
02: max_errors (i32) = 100,
03: disable_codegen (bool) = false,
04: batch_size (i32) = 0,
05: num_nodes (i32) = 0,
06: max_scan_range_length (i64) = 0,
07: num_scanner_threads (i32) = 0,
08: max_io_buffers (i32) = 0,
09: allow_unsupported_formats (bool) = false,
10: default_order_by_limit (i64) = -1,
11: debug_action (string) = "",
12: mem_limit (i64) = 0,
13: abort_on_default_limit_exceeded (bool) = false,
15: hbase_caching (i32) = 0,
16: hbase_cache_blocks (bool) = false,
17: parquet_file_size (i64) = 0,
18: explain_level (i32) = 1,
19: sync_ddl (bool) = false,
20: request_pool (string) = "impreports",
23: disable_cached_reads (bool) = false,
24: disable_outermost_topn (bool) = false,
25: rm_initial_mem (i64) = 0,
26: query_timeout_s (i32) = 0,
28: appx_count_distinct (bool) = false,
29: disable_unsafe_spills (bool) = false,
31: exec_single_node_rows_threshold (i32) = 100,
32: optimize_partition_key_scans (bool) = false,
33: replica_preference (i32) = 0,
34: schedule_random_replica (bool) = false,
35: scan_node_codegen_threshold (i64) = 1800000,
36: disable_streaming_preaggregations (bool) = false,
37: runtime_filter_mode (i32) = 2,
38: runtime_bloom_filter_size (i32) = 1048576,
39: runtime_filter_wait_time_ms (i32) = 0,
40: disable_row_runtime_filtering (bool) = false,
41: max_num_runtime_filters (i32) = 10,
42: parquet_annotate_strings_utf8 (bool) = false,
43: parquet_fallback_schema_resolution (i32) = 0,
45: s3_skip_insert_staging (bool) = true,
46: runtime_filter_min_size (i32) = 1048576,
47: runtime_filter_max_size (i32) = 16777216,
48: prefetch_mode (i32) = 1,
49: strict_mode (bool) = false,
50: scratch_limit (i64) = -1,
51: enable_expr_rewrites (bool) = true,
52: decimal_v2 (bool) = false,
53: parquet_dictionary_filtering (bool) = true,
54: parquet_array_resolution (i32) = 2,
55: parquet_read_statistics (bool) = true,
56: default_join_distribution_mode (i32) = 0,
57: disable_codegen_rows_threshold (i32) = 50000,
58: default_spillable_buffer_size (i64) = 2097152,
59: min_spillable_buffer_size (i64) = 65536,
60: max_row_size (i64) = 524288,
}
I0208 01:07:06.324410 7154 Frontend.java:909] Compiling query: select count(*) from dv_entersiteevent_prq_remote where year=2017 and month=10
I0208 01:07:06.327203 7154 Frontend.java:948] Compiled query.
I0208 01:07:06.359748 7154 admission-controller.cc:510] Schedule for id=5f49e927591a9042:faf8f28600000000 in pool_name=root.impreports cluster_mem_needed=105.00 MB PoolConfig: max_requests=-1 max_queued=200 max_mem=-1.00 B
I0208 01:07:06.359776 7154 admission-controller.cc:515] Stats: agg_num_running=0, agg_num_queued=0, agg_mem_reserved=0, local_host(local_mem_admitted=0, num_admitted_running=0, num_queued=0, backend_mem_reserved=0)
I0208 01:07:06.359784 7154 admission-controller.cc:531] Admitted query id=5f49e927591a9042:faf8f28600000000
I0208 01:07:06.359807 7154 coordinator.cc:99] Exec() query_id=5f49e927591a9042:faf8f28600000000 stmt=select count(*) from dv_entersiteevent_prq_remote where year=2017 and month=10
I0208 01:07:06.360694 7154 coordinator.cc:357] starting execution on 5 backends for query 5f49e927591a9042:faf8f28600000000
I0208 01:07:06.363255 63583 impala-internal-service.cc:44] ExecQueryFInstances(): query_id=5f49e927591a9042:faf8f28600000000
I0208 01:07:06.363282 63583 query-exec-mgr.cc:46] StartQueryFInstances() query_id=5f49e927591a9042:faf8f28600000000 coord=aopr-dhc001.lpdomain.com:22000
I0208 01:07:06.363296 63583 query-state.cc:173] Buffer pool limit for 5f49e927591a9042:faf8f28600000000: 10802010521
I0208 01:07:06.363364 63583 initial-reservations.cc:60] Successfully claimed initial reservations (0) for query 5f49e927591a9042:faf8f28600000000
I0208 01:07:06.363544 7155 query-state.cc:286] StartFInstances(): query_id=5f49e927591a9042:faf8f28600000000 #instances=2
I0208 01:07:06.363844 7155 query-state.cc:299] descriptor table for query=5f49e927591a9042:faf8f28600000000
tuples:
Tuple(id=2 size=0 slots=[] tuple_path=[])
Tuple(id=1 size=8 slots=[Slot(id=2 type=BIGINT col_path=[] offset=0 null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[])
Tuple(id=0 size=8 slots=[Slot(id=3 type=BIGINT col_path=[] offset=0 null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[])
I0208 01:07:06.364013 7156 query-state.cc:377] Executing instance. instance_id=5f49e927591a9042:faf8f28600000000 fragment_idx=0 per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=1
I0208 01:07:06.364135 7157 query-state.cc:377] Executing instance. instance_id=5f49e927591a9042:faf8f28600000005 fragment_idx=1 per_fragment_instance_idx=4 coord_state_idx=0 #in-flight=2
I0208 01:07:06.364241 7157 hdfs-scan-node.cc:160] Max row batch queue size for scan node '0' in fragment instance '5f49e927591a9042:faf8f28600000005': 160
I0208 01:07:06.366775 7154 coordinator.cc:370] started execution on 5 backends for query 5f49e927591a9042:faf8f28600000000
I0208 01:07:06.385625 7155 query-exec-mgr.cc:149] ReleaseQueryState(): query_id=5f49e927591a9042:faf8f28600000000 refcnt=4
I0208 01:07:06.671193 7559 impala-server.cc:1796] Connection from client ::ffff:10.16.136.3:55598 closed, closing 1 associated session(s)
I0208 01:07:06.676138 544 data-stream-mgr.cc:228] Reduced stream ID cache from 2 items, to 0, eviction took: 0
I0208 01:07:06.677202 544 coordinator.cc:955] Backend completed: host=aopr-dhc002.lpdomain.com:22000 remaining=4
I0208 01:07:06.677214 544 coordinator-backend-state.cc:229] query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: aopr-dhc001.lpdomain.com:22000
I0208 01:07:06.684221 7157 query-state.cc:384] Instance completed. instance_id=5f49e927591a9042:faf8f28600000005 #in-flight=1 status=OK
I0208 01:07:06.684237 7157 query-exec-mgr.cc:149] ReleaseQueryState(): query_id=5f49e927591a9042:faf8f28600000000 refcnt=3
I0208 01:07:06.700275 520 coordinator.cc:955] Backend completed: host=aopr-dhc005.lpdomain.com:22000 remaining=3
I0208 01:07:06.700289 520 coordinator-backend-state.cc:229] query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: aopr-dhc001.lpdomain.com:22000
I0208 01:07:06.704620 65258 coordinator.cc:955] Backend completed: host=aopr-dhc003.lpdomain.com:22000 remaining=2
I0208 01:07:06.704666 65258 coordinator-backend-state.cc:229] query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: aopr-dhc001.lpdomain.com:22000
I0208 01:07:06.706041 7156 data-stream-mgr.cc:238] DeregisterRecvr(): fragment_instance_id=5f49e927591a9042:faf8f28600000000, node=2
I0208 01:07:06.706070 7156 data-stream-recvr.cc:235] cancelled stream: fragment_instance_id_=5f49e927591a9042:faf8f28600000000 node_id=2
I0208 01:07:06.706809 64916 coordinator.cc:955] Backend completed: host=aopr-dhc004.lpdomain.com:22000 remaining=1
I0208 01:07:06.706821 64916 coordinator-backend-state.cc:229] query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: aopr-dhc001.lpdomain.com:22000
I0208 01:07:06.789568 7154 impala-beeswax-server.cc:193] get_results_metadata(): query_id=5f49e927591a9042:faf8f28600000000
I0208 01:07:06.790439 7154 coordinator.cc:789] Coordinator waiting for backends to finish, 1 remaining
I0208 01:07:06.791052 7154 coordinator.cc:794] All backends finished successfully.
I0208 01:07:06.791086 7154 coordinator.cc:1090] Release admssion control resources for query 5f49e927591a9042:faf8f28600000000
I0208 01:07:06.791363 7156 query-state.cc:384] Instance completed. instance_id=5f49e927591a9042:faf8f28600000000 #in-flight=0 status=OK
I0208 01:07:06.791414 7156 query-exec-mgr.cc:149] ReleaseQueryState(): query_id=5f49e927591a9042:faf8f28600000000 refcnt=2
I0208 01:07:06.793586 7154 impala-beeswax-server.cc:235] close(): query_id=5f49e927591a9042:faf8f28600000000
I0208 01:07:06.793602 7154 impala-server.cc:992] UnregisterQuery(): query_id=5f49e927591a9042:faf8f28600000000
I0208 01:07:06.793614 7154 impala-server.cc:1075] Cancel(): query_id=5f49e927591a9042:faf8f28600000000
I0208 01:07:06.793624 7154 coordinator.cc:895] Cancel() query_id=5f49e927591a9042:faf8f28600000000
I0208 01:07:06.793637 7154 coordinator.cc:905] CancelBackends() query_id=5f49e927591a9042:faf8f28600000000, tried to cancel 0 backends
I0208 01:07:06.802776 7154 query-exec-mgr.cc:149] ReleaseQueryState(): query_id=5f49e927591a9042:faf8f28600000000 refcnt=1
I0208 01:07:06.814214 7154 impala-server.cc:1796] Connection from client ::ffff:10.16.144.131:51012 closed, closing 1 associated session(s)
I0208 01:07:07.258891 7567 impala-server.cc:1796] Connection from client ::ffff:10.16.136.2:56375 closed, closing 1 associated session(s)
I0208 01:07:11.600131 7819 impala-server.cc:1796] Connection from client ::ffff:10.16.136.3:27460 closed, closing 1 associated session(s)
I0208 01:07:12.262739 7821 impala-server.cc:1796] Connection from client ::ffff:10.16.136.2:52672 closed, closing 1 associated session(s)
I0208 01:07:16.665451 7822 impala-server.cc:1796] Connection from client ::ffff:10.16.136.3:10693 closed, closing 1 associated session(s)
I0208 01:07:17.235321 7825 impala-server.cc:1796] Connection from client ::ffff:10.16.136.2:34752 closed, 

1 REPLY 1
Highlighted

Re: impala submission control issue

Cloudera Employee

Hello Fawze,

 

Not sure if you are still experiencing the same issue for now. This issue was resolved from CDH 5.15. 

 

As a workaround for the CDH version before 5.15, I would suggest you to add any non existing user in Default pool settings.
## CM->Clusters->Dynamic Resource Pool Configuration->Impala Admission Control-> Default Settings->Check "Allow these users and groups to submit to this pool"->
Add any non existing user under "Users". Like "<non_existing_user>"
Refresh Dynamic resource pool.


And try to re run the impala query and check if it still throwing the error or not.

 

Thanks

Benny