Reply
Explorer
Posts: 28
Registered: ‎07-17-2017

Conflicts between DDL operations(ALTER..) and SELECT.. -Missing tables were not received in 120000ms

hi,

When I excecute any 2 queries concurently :

1- ALTER(DROP) table1 ..
2- SELECT * FROM table2 .. / INSERT..

Impalad log file:

W0929 09:40:11.309067 28378 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
W0929 09:42:11.316686 28378 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
W0929 09:44:11.323714 28378 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
W0929 09:46:11.330497 28378 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
W0929 09:48:11.337724 28378 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
W0929 09:50:11.344326 28378 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.


NB: I use Impala Shell v2.9.0-cdh5.12.0 (03c6ddb) built on Thu Jun 29 04:17:31 PDT 2017

Thnaks in advance.

Explorer
Posts: 28
Registered: ‎07-17-2017

Re: Conflicts between DDL operations(ALTER..) and SELECT.. -Missing tables were not received in 1200

Cloudera Employee
Posts: 158
Registered: ‎03-23-2015

Re: Conflicts between DDL operations(ALTER..) and SELECT.. -Missing tables were not received in 1200

Hi,

I got your email and have a quick look at the issue, I have a couple of questions:

1. if you run them separately, will query work? From your issue description, it should, but I would like to confirm

2. any clues from impala daemon and catalogd server logs? From the error message, it looks like the metadata for those tables are big and impala daemon was not able to get the full table metadata in time.

So if you can upload full logs to IMPALA-5996, it would be useful.

Please also check if you can find any similar messages from https://issues.apache.org/jira/browse/IMPALA-2648.

3. can you please also share the full query that you ran for both of them?
Explorer
Posts: 28
Registered: ‎07-17-2017

Re: Conflicts between DDL operations(ALTER..) and SELECT.. -Missing tables were not received in 1200

[ Edited ]

Hi,
Thank you @EricL for the answer,

1- The DML operations always work, and for DDL on big number of partitions tables the COMPUTE STATS work but the DROP give me this problem (https://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Can-not-ALTER-or-DROP-a-big-partitionn...)

2- Here is the Catalogd, Metastore and Metastore logs :

Impalad :

I1004 09:14:17.082247 29871 Frontend.java:892] Compiling query: select MAX(created_at) FROM table_2
I1004 09:14:17.082911 29871 Frontend.java:834] Requesting prioritized load of table(s): table_2
W1004 09:16:17.097311 29871 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
I1004 09:16:17.103250 29871 Frontend.java:834] Requesting prioritized load of table(s): table_2
W1004 09:18:17.114742 29871 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
I1004 09:18:17.115595 29871 Frontend.java:834] Requesting prioritized load of table(s): table_2
W1004 09:20:17.128476 29871 Frontend.java:917] Missing tables were not received in 120000ms. Load request will be retried.
I1004 09:20:17.129413 29871 Frontend.java:834] Requesting prioritized load of table(s): table_2

 

Catalogd :

I1004 09:14:17.077718 42148 TableLoadingMgr.java:285] Loading next table from queue: table_2
I1004 09:14:17.077848 42148 TableLoadingMgr.java:287] Remaining items in queue: 0. Loads in progress: 0


Metastore
:

2017-10-04 09:16:33,583 WARN  hive.log: [pool-5-thread-6]: Updating partition stats fast for: table_1
2017-10-04 09:16:33,585 WARN  hive.log: [pool-5-thread-6]: Updated size to 4353
2017-10-04 09:16:33,603 WARN  hive.log: [pool-5-thread-6]: Updating partition stats fast for: table_1
2017-10-04 09:16:33,604 WARN  hive.log: [pool-5-thread-6]: Updated size to 4094
2017-10-04 09:16:33,623 WARN  hive.log: [pool-5-thread-6]: Updating partition stats fast for: table_1
2017-10-04 09:16:33,624 WARN  hive.log: [pool-5-thread-6]: Updated size to 12424
2017-10-04 09:16:33,643 WARN  hive.log: [pool-5-thread-6]: Updating partition stats fast for: table_1
2017-10-04 09:16:33,644 WARN  hive.log: [pool-5-thread-6]: Updated size to 2831


3- Queries
:
Q1: compute stats table_1; [37K partitions].
Q2: select MAX(created_at) FROM table_2; [1 partition/ 1M rows].

NB: I updated in https://issues.apache.org/jira/browse/IMPALA-5996.

Thanks again for you help.

Announcements