Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error while fetching data from impala in qliksense server

Highlighted

Error while fetching data from impala in qliksense server

New Contributor

While trying to fetch data from impala using qliksense , job fails after some point in qlik. The size of table was comparatively less (~ 2 GB) .This error was observed as the size of data is increasing incrementally. Not sure why is this occuring ? Any help is appreciatable

 

I0922 09:29:26.826395  1596 plan-fragment-executor.cc:327] Open(): instance_id=924e59e87739aa6b:fd8134b800000002
I0922 09:29:26.851933  1597 plan-fragment-executor.cc:327] Open(): instance_id=d149335a92fea933:1d77299700000002
I0922 09:30:23.271356  6838 impala-server.cc:1006] Cancel(): query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.271373  6838 coordinator.cc:1435] Cancel() query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.271378  6838 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.271380  6838 plan-fragment-executor.cc:544] Cancel(): instance_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.271384  6838 data-stream-mgr.cc:258] cancelling all streams for fragment=d149335a92fea933:1d77299700000000
I0922 09:30:23.271389  6838 data-stream-recvr.cc:237] cancelled stream: fragment_instance_id_=d149335a92fea933:1d77299700000000 node_id=2
I0922 09:30:23.271399  6838 data-stream-recvr.cc:237] cancelled stream: fragment_instance_id_=d149335a92fea933:1d77299700000000 node_id=2
I0922 09:30:23.271412  6838 data-stream-recvr.cc:237] cancelled stream: fragment_instance_id_=d149335a92fea933:1d77299700000000 node_id=2
I0922 09:30:23.271428  6838 coordinator.cc:1484] sending CancelPlanFragment rpc for instance_id=d149335a92fea933:1d77299700000001 backend=ip-152-0-1-71.eu-west-1.compute.internal:22000
I0922 09:30:23.271893  6838 coordinator.cc:1484] sending CancelPlanFragment rpc for instance_id=d149335a92fea933:1d77299700000002 backend=ip-152-0-1-113.eu-west-1.compute.internal:22000
I0922 09:30:23.272022 12862 fragment-mgr.cc:129] CancelPlanFragment(): instance_id=d149335a92fea933:1d77299700000002
I0922 09:30:23.272058 12862 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.272063 12862 plan-fragment-executor.cc:544] Cancel(): instance_id=d149335a92fea933:1d77299700000002
I0922 09:30:23.272068 12862 data-stream-mgr.cc:258] cancelling all streams for fragment=d149335a92fea933:1d77299700000002
I0922 09:30:23.272109  6838 coordinator.cc:1484] sending CancelPlanFragment rpc for instance_id=d149335a92fea933:1d77299700000003 backend=ip-152-0-1-181.eu-west-1.compute.internal:22000
I0922 09:30:23.276639  6965 impala-server.cc:920] UnregisterQuery(): query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.276685  6965 impala-server.cc:1006] Cancel(): query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.277007  6838 impala-beeswax-server.cc:352] close(): query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.277014  6838 impala-server.cc:920] UnregisterQuery(): query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.277022  6838 impala-server.cc:1006] Cancel(): query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.279839  6965 data-stream-mgr.cc:226] DeregisterRecvr(): fragment_instance_id=d149335a92fea933:1d77299700000000, node=2
I0922 09:30:23.286447  6838 status.cc:114] Invalid or unknown query handle
    @           0x850589  (unknown)
    @           0xad0f2b  (unknown)
    @           0xadedf1  (unknown)
    @           0xb1ca1d  (unknown)
    @           0xd58295  (unknown)
    @           0xd5faf4  (unknown)
    @           0x81de8c  (unknown)
    @          0x1b5ad8b  (unknown)
    @          0x1b428d9  (unknown)
    @           0xa10d19  (unknown)
    @           0xa11772  (unknown)
    @           0xbfc0c9  (unknown)
    @           0xbfca64  (unknown)
    @           0xe665da  (unknown)
    @     0x7f49bb42adc5  start_thread
    @     0x7f49bb15973d  __clone
I0922 09:30:23.288688  6838 impala-server.cc:1682] Connection from client 152.0.1.238:38092 closed, closing 1 associated session(s)
I0922 09:30:23.297158  6838 status.cc:114] Session closed
    @           0x850589  (unknown)
    @           0xadf5d3  (unknown)
    @           0xadfb28  (unknown)
    @           0xa0a960  (unknown)
    @          0x1b5b863  (unknown)
    @          0x1b428d9  (unknown)
    @           0xa10d19  (unknown)
    @           0xa11772  (unknown)
    @           0xbfc0c9  (unknown)
    @           0xbfca64  (unknown)
    @           0xe665da  (unknown)
    @     0x7f49bb42adc5  start_thread
    @     0x7f49bb15973d  __clone
I0922 09:30:23.348542  6965 impala-hs2-server.cc:985] CloseOperation(): query_id=d149335a92fea933:1d77299700000000
I0922 09:30:23.401504  6965 impala-server.cc:1682] Connection from client 152.0.4.118:45357 closed, closing 1 associated session(s)
I0922 09:30:23.408830 12902 fragment-mgr.cc:129] CancelPlanFragment(): instance_id=a044b29b6b557676:152d1c7300000002
I0922 09:30:23.408845 12902 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.408850 12902 plan-fragment-executor.cc:544] Cancel(): instance_id=a044b29b6b557676:152d1c7300000002
I0922 09:30:23.408857 12902 data-stream-mgr.cc:258] cancelling all streams for fragment=a044b29b6b557676:152d1c7300000002
I0922 09:30:23.411087  6965 status.cc:114] Session closed
    @           0x850589  (unknown)
    @           0xadf5d3  (unknown)
    @           0xadfb28  (unknown)
    @           0xa0a960  (unknown)
    @          0x1b5b863  (unknown)
    @          0x1b428d9  (unknown)
    @           0xa10d19  (unknown)
    @           0xa11772  (unknown)
    @           0xbfc0c9  (unknown)
    @           0xbfca64  (unknown)
    @           0xe665da  (unknown)
    @     0x7f49bb42adc5  start_thread
    @     0x7f49bb15973d  __clone
I0922 09:30:23.419782  7834 fragment-mgr.cc:129] CancelPlanFragment(): instance_id=e24c84a8e0850d34:e6127d0d00000002
I0922 09:30:23.419797  7834 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.419800  7834 plan-fragment-executor.cc:544] Cancel(): instance_id=e24c84a8e0850d34:e6127d0d00000002
I0922 09:30:23.419805  7834 data-stream-mgr.cc:258] cancelling all streams for fragment=e24c84a8e0850d34:e6127d0d00000002
I0922 09:30:23.419991  1421 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.420055  1421 plan-fragment-executor.cc:544] Cancel(): instance_id=a044b29b6b557676:152d1c7300000002
I0922 09:30:23.420063  1421 data-stream-mgr.cc:258] cancelling all streams for fragment=a044b29b6b557676:152d1c7300000002
I0922 09:30:23.434639  1557 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.434645  1557 plan-fragment-executor.cc:544] Cancel(): instance_id=e24c84a8e0850d34:e6127d0d00000002
I0922 09:30:23.434650  1557 data-stream-mgr.cc:258] cancelling all streams for fragment=e24c84a8e0850d34:e6127d0d00000002
I0922 09:30:23.443321  7889 fragment-mgr.cc:129] CancelPlanFragment(): instance_id=9545f4f62f855981:f05c998c00000002
I0922 09:30:23.443439  7889 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.443442  7889 plan-fragment-executor.cc:544] Cancel(): instance_id=9545f4f62f855981:f05c998c00000002
I0922 09:30:23.443446  7889 data-stream-mgr.cc:258] cancelling all streams for fragment=9545f4f62f855981:f05c998c00000002
I0922 09:30:23.453485 12902 fragment-mgr.cc:129] CancelPlanFragment(): instance_id=924e59e87739aa6b:fd8134b800000002
I0922 09:30:23.453495 12902 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.453497 12902 plan-fragment-executor.cc:544] Cancel(): instance_id=924e59e87739aa6b:fd8134b800000002
I0922 09:30:23.453500 12902 data-stream-mgr.cc:258] cancelling all streams for fragment=924e59e87739aa6b:fd8134b800000002
I0922 09:30:23.461135  1581 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.461141  1581 plan-fragment-executor.cc:544] Cancel(): instance_id=9545f4f62f855981:f05c998c00000002
I0922 09:30:23.461145  1581 data-stream-mgr.cc:258] cancelling all streams for fragment=9545f4f62f855981:f05c998c00000002
I0922 09:30:23.468094  1596 plan-fragment-executor.cc:536] Cancelling plan fragment...
I0922 09:30:23.468102  1596 plan-fragment-executor.cc:544] Cancel(): instance_id=924e59e87739aa6b:fd8134b800000002
I0922 09:30:23.468107  1596 data-stream-mgr.cc:258] cancelling all streams for fragment=924e59e87739aa6b:fd8134b800000002

.

 

These are the logs from the impala daemon on which the job was triggered.  

4 REPLIES 4

Re: Error while fetching data from impala in qliksense server

Master Collaborator

What error was reported back to the client? What version of Impala? Do you have a profile for the query from cloudera manager or the impala daemon debug page (http://impalad-hostname:25000/queries)? How long does the query run for before failing?

 

One possibility is that the session or query is timing out, but I don't really know without more information.

Re: Error while fetching data from impala in qliksense server

New Contributor

Thank you for your response Tim. The below mentioned details will povide you inputs required:

1.Qliksense Server (client) responded as  
"Unable to get column information for the fields that are used in the query: The connection has been disabled."

2.Impala version : 2.7.0

3.Error in impala daemon debug page:

Query (id=6f49510f4baca8b7:6aa9721c00000000):
Summary:
Session ID: d34977353b25284e:b3c290911a303690
Session Type: HIVESERVER2
HiveServer2 Protocol Version: V6
Start Time: 2017-09-25 06:21:25.478093000
End Time:
Query Type: QUERY
Query State: CREATED
Query Status: OK
Impala Version: impalad version 2.7.0-cdh5.9.1 RELEASE (build 24ad6df788d66e4af9496edb26ac4d1f1d2a1f2c)
User: 147820
Connected User: 147820
Delegated User:
Network Address: 152.0.1.87:49730
Default Db: domain_layer
Sql Statement: select * from transform_layer.DOM_FTE c where c.account='16011975275'
Coordinator: ip-152-0-1-113.eu-west-1.compute.internal:22000
Query Options (non default): QUERY_TIMEOUT_S=600
Plan:
----------------
Estimated Per-Host Requirements: Memory=64.00MB VCores=1
WARNING: The following tables are missing relevant table and/or column statistics.
transform_layer.dom_fte

01:EXCHANGE [UNPARTITIONED]
| hosts=3 per-host-mem=unavailable
| tuple-ids=0 row-size=23B cardinality=316278
|
00:SCAN HDFS [transform_layer.dom_fte c, RANDOM]
partitions=1/1 files=9 size=42.84MB
predicates: c.account = '16011975275'
table stats: 3162781 rows total
column stats: unavailable
hosts=3 per-host-mem=64.00MB
tuple-ids=0 row-size=23B cardinality=316278
----------------
Estimated Per-Host Mem: 67108864
Estimated Per-Host VCores: 1
Tables Missing Stats: transform_layer.dom_fte
Request Pool: root.147820
Admission result: Admitted immediately
Planner Timeline: 6.061ms
- Analysis finished: 1.175ms (1.175ms)
- Equivalence classes computed: 1.321ms (146.615us)
- Single node plan created: 2.745ms (1.423ms)
- Runtime filters computed: 2.774ms (29.373us)
- Distributed plan created: 3.359ms (584.729us)
- Lineage info computed: 4.187ms (827.857us)
- Planning finished: 6.061ms (1.874ms)
Query Timeline: 34m6s
- Start execution: 0.000ns (0.000ns)
- Planning finished: 5.999ms (5.999ms)
- Submit for admission: 6.999ms (999.989us)
- Completed admission: 6.999ms (0.000ns)
- Ready to start 3 remote fragments: 6.999ms (0.000ns)
- All 3 remote fragments started: 7.999ms (999.990us)
- Rows available: 127.998ms (119.998ms)
- First row fetched: 569.994ms (441.995ms)
- ComputeScanRangeAssignmentTimer: 0.000ns
ImpalaServer:
- ClientFetchWaitTimer: 1s147ms
- RowMaterializationTimer: 0.000ns
Execution Profile 6f49510f4baca8b7:6aa9721c00000000:(Total: 120.998ms, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 0
Filter routing table:
ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled
----------------------------------------------------------------------------------------------------------------------------

Fragment start latencies: Count: 3, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms
Per Node Peak Memory Usage: ip-152-0-1-181.eu-west-1.compute.internal:22000(32.14 MB) ip-152-0-1-71.eu-west-1.compute.internal:22000(32.14 MB) ip-152-0-1-113.eu-west-1.compute.internal:22000(32.15 MB)
- FiltersReceived: 0 (0)
- FinalizationTimer: 0.000ns
Coordinator Fragment F01:(Total: 119.998ms, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 8.12 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 12.01 KB (12296)
- PerHostPeakMemUsage: 0
- PrepareTime: 0.000ns
- RowsProduced: 0 (0)
- TotalCpuTime: 442.995ms
- TotalNetworkReceiveTime: 119.998ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 0 (0)
- BlocksRecycled: 0 (0)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 12.80 GB (13743895552)
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0.000ns
- TotalEncryptionTime: 0.000ns
- TotalIntegrityCheckTime: 0.000ns
- TotalReadBlockTime: 0.000ns
EXCHANGE_NODE (id=1):(Total: 119.998ms, non-child: 0.000ns, % non-child: 0.00%)
BytesReceived(32s000ms): 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B, 47.00 B
- BytesReceived: 47.00 B (47)
- ConvertRowBatchTime: 0.000ns
- DeserializeRowBatchTimer: 0.000ns
- FirstBatchArrivalWaitTime: 119.998ms
- PeakMemoryUsage: 0
- RowsReturned: 1 (1)
- RowsReturnedRate: 8.00 /sec
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F00:(Total: 66.665ms, non-child: 0.000ns, % non-child: 0.00%)
split sizes: min: 14.28 MB, max: 14.28 MB, avg: 14.28 MB, stddev: 3.45 KB
completion times: min:119.998ms max:123.998ms mean: 121.998ms stddev:1.632ms
execution rates: min:115.20 MB/sec max:118.98 MB/sec mean:117.07 MB/sec stddev:1.54 MB/sec
num instances: 3
- AverageThreadTokens: 0.33
- BloomFilterBytes: 0
- PeakMemoryUsage: 32.14 MB (33701888)
- PerHostPeakMemUsage: 32.14 MB (33704618)
- PrepareTime: 33.332ms
- RowsProduced: 0 (0)
- TotalCpuTime: 187.663ms
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 20.666ms
CodeGen:(Total: 87.331ms, non-child: 87.331ms, % non-child: 100.00%)
- CodegenTime: 333.335us
- CompileTime: 14.333ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1984068)
- NumFunctions: 30 (30)
- NumInstructions: 511 (511)
- OptimizationTime: 39.999ms
- PrepareTime: 32.332ms
DataStreamSender (dst_id=1):
- BytesSent: 15.00 B (15)
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 17.00 B (17)
HDFS_SCAN_NODE (id=0):(Total: 98.998ms, non-child: 98.998ms, % non-child: 100.00%)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 0.00
- BytesRead: 14.28 MB (14973756)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 14.28 MB (14973756)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 14.28 MB (14973756)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumDisksAccessed: 2 (2)
- NumScannerThreadsStarted: 3 (3)
- PeakMemoryUsage: 32.13 MB (33689600)
- PerReadThreadRawHdfsThroughput: 1.83 GB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 1.05M (1054260)
- RowsReturned: 0 (0)
- RowsReturnedRate: 3.00 /sec
- ScanRangesComplete: 3 (3)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 86.331ms
- DelimiterParseTime: 48.999ms
- MaterializeTupleTime(*): 14.666ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 63.429ms
- ScannerThreadsVoluntaryContextSwitches: 15 (15)
- TotalRawHdfsReadTime(*): 7.666ms
- TotalReadThroughput: 0.00 /sec
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 0 (0)
- BlocksRecycled: 0 (0)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 12.80 GB (13743895552)
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0.000ns
- TotalEncryptionTime: 0.000ns
- TotalIntegrityCheckTime: 0.000ns
- TotalReadBlockTime: 0.000ns
Fragment F00:
Instance 6f49510f4baca8b7:6aa9721c00000002 (host=ip-152-0-1-71.eu-west-1.compute.internal:22000):(Total: 69.000ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/4.76 MB 1:2/9.52 MB
MemoryUsage(500.000ms): 0
ThreadUsage(500.000ms): 1
- AverageThreadTokens: 1.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 32.14 MB (33701888)
- PerHostPeakMemUsage: 32.14 MB (33701888)
- PrepareTime: 34.000ms
- RowsProduced: 0 (0)
- TotalCpuTime: 192.001ms
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 20.000ms
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 0 (0)
- BlocksRecycled: 0 (0)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 12.80 GB (13743895552)
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0.000ns
- TotalEncryptionTime: 0.000ns
- TotalIntegrityCheckTime: 0.000ns
- TotalReadBlockTime: 0.000ns
CodeGen:(Total: 88.000ms, non-child: 88.000ms, % non-child: 100.00%)
- CodegenTime: 1.000ms
- CompileTime: 14.000ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1984068)
- NumFunctions: 30 (30)
- NumInstructions: 511 (511)
- OptimizationTime: 41.000ms
- PrepareTime: 33.000ms
DataStreamSender (dst_id=1):
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
HDFS_SCAN_NODE (id=0):(Total: 101.000ms, non-child: 101.000ms, % non-child: 100.00%)
ExecOption: Expr Evaluation Codegen Enabled, TEXT Codegen Enabled, Codegen enabled: 3 out of 3
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/4.76 MB 1:2/9.52 MB
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0%
File Formats: TEXT/NONE:3
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 0.00
- BytesRead: 14.28 MB (14978747)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 14.28 MB (14978747)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 14.28 MB (14978747)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumDisksAccessed: 2 (2)
- NumScannerThreadsStarted: 3 (3)
- PeakMemoryUsage: 32.13 MB (33689600)
- PerReadThreadRawHdfsThroughput: 1.74 GB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 1.05M (1054578)
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
- ScanRangesComplete: 3 (3)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 88.000ms
- DelimiterParseTime: 52.000ms
- MaterializeTupleTime(*): 15.000ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 64.882ms
- ScannerThreadsVoluntaryContextSwitches: 12 (12)
- TotalRawHdfsReadTime(*): 8.000ms
- TotalReadThroughput: 0.00 /sec
Instance 6f49510f4baca8b7:6aa9721c00000003 (host=ip-152-0-1-181.eu-west-1.compute.internal:22000):(Total: 66.997ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:2/9.51 MB 1:1/4.77 MB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 32.14 MB (33701888)
- PerHostPeakMemUsage: 32.14 MB (33701888)
- PrepareTime: 32.998ms
- RowsProduced: 1 (1)
- TotalCpuTime: 186.991ms
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 21.999ms
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 0 (0)
- BlocksRecycled: 0 (0)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 12.80 GB (13743895552)
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0.000ns
- TotalEncryptionTime: 0.000ns
- TotalIntegrityCheckTime: 0.000ns
- TotalReadBlockTime: 0.000ns
CodeGen:(Total: 86.996ms, non-child: 86.996ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 13.999ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1984068)
- NumFunctions: 30 (30)
- NumInstructions: 511 (511)
- OptimizationTime: 39.998ms
- PrepareTime: 31.998ms
DataStreamSender (dst_id=1):
- BytesSent: 47.00 B (47)
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- RowsReturned: 1 (1)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 51.00 B (51)
HDFS_SCAN_NODE (id=0):(Total: 98.995ms, non-child: 98.995ms, % non-child: 100.00%)
ExecOption: Expr Evaluation Codegen Enabled, TEXT Codegen Enabled, Codegen enabled: 3 out of 3
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:2/9.51 MB 1:1/4.77 MB
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0%
File Formats: TEXT/NONE:3
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 0.00
- BytesRead: 14.28 MB (14971192)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 14.28 MB (14971192)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 14.28 MB (14971192)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumDisksAccessed: 2 (2)
- NumScannerThreadsStarted: 3 (3)
- PeakMemoryUsage: 32.13 MB (33689600)
- PerReadThreadRawHdfsThroughput: 1.74 GB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 1.05M (1054046)
- RowsReturned: 1 (1)
- RowsReturnedRate: 10.00 /sec
- ScanRangesComplete: 3 (3)
- ScannerThreadsInvoluntaryContextSwitches: 2 (2)
- ScannerThreadsTotalWallClockTime: 86.996ms
- DelimiterParseTime: 49.997ms
- MaterializeTupleTime(*): 13.999ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 64.712ms
- ScannerThreadsVoluntaryContextSwitches: 18 (18)
- TotalRawHdfsReadTime(*): 7.999ms
- TotalReadThroughput: 0.00 /sec
Instance 6f49510f4baca8b7:6aa9721c00000001 (host=ip-152-0-1-113.eu-west-1.compute.internal:22000):(Total: 63.999ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:2/9.52 MB 0:1/4.75 MB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 32.14 MB (33701888)
- PerHostPeakMemUsage: 32.15 MB (33710080)
- PrepareTime: 32.999ms
- RowsProduced: 0 (0)
- TotalCpuTime: 183.998ms
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 19.999ms
CodeGen:(Total: 86.999ms, non-child: 86.999ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 14.999ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1984068)
- NumFunctions: 30 (30)
- NumInstructions: 511 (511)
- OptimizationTime: 38.999ms
- PrepareTime: 31.999ms
DataStreamSender (dst_id=1):
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
HDFS_SCAN_NODE (id=0):(Total: 96.998ms, non-child: 96.998ms, % non-child: 100.00%)
ExecOption: Expr Evaluation Codegen Enabled, TEXT Codegen Enabled, Codegen enabled: 3 out of 3
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:2/9.52 MB 0:1/4.75 MB
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0%
File Formats: TEXT/NONE:3
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 0.00
- BytesRead: 14.28 MB (14971329)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 14.28 MB (14971329)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 14.28 MB (14971329)
- DecompressionTime: 0.000ns
- MaxCompressedTextFileLength: 0
- NumDisksAccessed: 2 (2)
- NumScannerThreadsStarted: 3 (3)
- PeakMemoryUsage: 32.13 MB (33689600)
- PerReadThreadRawHdfsThroughput: 1.99 GB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 1.05M (1054157)
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
- ScanRangesComplete: 3 (3)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 83.999ms
- DelimiterParseTime: 44.999ms
- MaterializeTupleTime(*): 14.999ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 60.694ms
- ScannerThreadsVoluntaryContextSwitches: 16 (16)
- TotalRawHdfsReadTime(*): 6.999ms
- TotalReadThroughput: 0.00 /sec

4.The query ran for 5 mins. and 45 sec before giving any error.

Re: Error while fetching data from impala in qliksense server

Master Collaborator

It looks like your query hit the query timeout - query_timeout_s is set to 600 seconds. It looks like the query executed quickly (it finished scanning all 9 files and found 1 row) but the client didn't fetch all the results or close the query. I suspect the problem is on the client. I think that error message is produced by Qliksense but I don't know what conditions would trigger that error condition in Qliksense.

Re: Error while fetching data from impala in qliksense server

New Contributor

Can the query timeout be increased? Will it resolve the issue?