Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Partition Spec Does Not Exist error

Solved Go to solution

Partition Spec Does Not Exist error

New Contributor

Hi there,

 

We're seeing an intermittent issue with some of our Impala queries, where a compute incremental stats statement will fail with an "AnalysisException: Partition spec does not exist:" error, immediately after we have created the partition in the same script.  So, a (faked) example would look like:

 

ALTER TABLE db.table_name DROP IF EXISTS PARTITION (partition_column = 1234);

INSERT INTO TABLE 
   db.table_name 
PARTITION 
(
   partition_column = 1234
)
SELECT   
   column1
 , column2
 , ...
FROM
   db.another_table
;

COMPUTE INCREMENTAL STATS db.table_name (partition_column = 1234);

AnalysisException: Partition spec does not exist: (partition_column = 1234).

 

The tables in question are not being modifed by any external processes so we're not sure what could be happening here - has anyone else encountered this issue?

 

Thanks,

Mike

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Partition Spec Does Not Exist error

Contributor

This is a known issue, but I havent yet seen it without multiple concurent Impala writers.

(I have been able to repro it both in the same impalad and in different impalads.)

 

https://issues.cloudera.org/browse/IMPALA-2423

 

Is there only a single writer using a single impalad at the time?

6 REPLIES 6

Re: Partition Spec Does Not Exist error

Contributor

This is a known issue, but I havent yet seen it without multiple concurent Impala writers.

(I have been able to repro it both in the same impalad and in different impalads.)

 

https://issues.cloudera.org/browse/IMPALA-2423

 

Is there only a single writer using a single impalad at the time?

Re: Partition Spec Does Not Exist error

New Contributor

Thanks for the response, and  yes, is looks like that's the issue - going over our logs it sems that this only occurs when we have multiple impalads targeting different partitions in the same table.

 

Incidentally, I think we've seen this with as few as two concurrent operations, but in this case the two operations are likely to both start and complete at almost exactly the same times, so I guess there may be some race condition when updating the partition metadata?

 

 

Re: Partition Spec Does Not Exist error

Contributor

Could be. We're still looking into it.

Re: Partition Spec Does Not Exist error

Explorer

Hi jbapple,

I'm a colleague of Mike that originally submitted this. we're still hitting the issue  (Impala v2.3.0-cdh5.5.1), any update on whether or not this might be fixed?

 

regards

Jamie

Re: Partition Spec Does Not Exist error

Contributor

You can follow the bug I linked to above for any future updates. Right nowI have no news for you.

Highlighted

Re: Partition Spec Does Not Exist error

Explorer

@jbapple wrote:

You can follow the bug I linked to above for any future updates. Right nowI have no news for you.


DOH! Didn't even see that link. Must pay more attention, my apologies.

Don't have an account?
Coming from Hortonworks? Activate your account here