Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Partition Spec Does Not Exist error

avatar
New Contributor

Hi there,

 

We're seeing an intermittent issue with some of our Impala queries, where a compute incremental stats statement will fail with an "AnalysisException: Partition spec does not exist:" error, immediately after we have created the partition in the same script.  So, a (faked) example would look like:

 

ALTER TABLE db.table_name DROP IF EXISTS PARTITION (partition_column = 1234);

INSERT INTO TABLE 
   db.table_name 
PARTITION 
(
   partition_column = 1234
)
SELECT   
   column1
 , column2
 , ...
FROM
   db.another_table
;

COMPUTE INCREMENTAL STATS db.table_name (partition_column = 1234);

AnalysisException: Partition spec does not exist: (partition_column = 1234).

 

The tables in question are not being modifed by any external processes so we're not sure what could be happening here - has anyone else encountered this issue?

 

Thanks,

Mike

 

 

1 ACCEPTED SOLUTION

avatar
Rising Star

This is a known issue, but I havent yet seen it without multiple concurent Impala writers.

(I have been able to repro it both in the same impalad and in different impalads.)

 

https://issues.cloudera.org/browse/IMPALA-2423

 

Is there only a single writer using a single impalad at the time?

View solution in original post

6 REPLIES 6

avatar
Rising Star

This is a known issue, but I havent yet seen it without multiple concurent Impala writers.

(I have been able to repro it both in the same impalad and in different impalads.)

 

https://issues.cloudera.org/browse/IMPALA-2423

 

Is there only a single writer using a single impalad at the time?

avatar
New Contributor

Thanks for the response, and  yes, is looks like that's the issue - going over our logs it sems that this only occurs when we have multiple impalads targeting different partitions in the same table.

 

Incidentally, I think we've seen this with as few as two concurrent operations, but in this case the two operations are likely to both start and complete at almost exactly the same times, so I guess there may be some race condition when updating the partition metadata?

 

 

avatar
Rising Star

Could be. We're still looking into it.

avatar
Explorer

Hi jbapple,

I'm a colleague of Mike that originally submitted this. we're still hitting the issue  (Impala v2.3.0-cdh5.5.1), any update on whether or not this might be fixed?

 

regards

Jamie

avatar
Rising Star

You can follow the bug I linked to above for any future updates. Right nowI have no news for you.

avatar
Explorer

@jbapple wrote:

You can follow the bug I linked to above for any future updates. Right nowI have no news for you.


DOH! Didn't even see that link. Must pay more attention, my apologies.