Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executing..

Solved Go to solution
Highlighted

Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executing..

Expert Contributor

Hi,

I have a partitionned tables that have more than 50k partitions, it work a good except the Hive Metastore operations, like DROP and ALTER ... RENAME, I face this error message:

Query: drop table cars
ERROR: ImpalaRuntimeException: Error making 'dropTable' RPC to Hive Metastore: 
CAUSED BY: MetaException: Timeout when executing method: drop_table_with_environment_context; 600003ms exceeds 600000ms

I don't now if it's, a memory problem, or it's normal and I should adjust the timeout value.. ? if yes, which one ?

Thanks to help me ASAP.

3 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Guru
Hi,

This is a known issue and we have an internal JIRA to track it. Current there is no better way to improve the performance.

You can, however, to increase the timeout limit via "hive.metastore.client.socket.timeout", and set it to long time value (in seconds) to allow the query to finish. It should finish eventually, but might take sometime.

Our engineers are still discussing internally to see the best fix for the issue, however, it is on going and we do not have solution yet at this stage.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Guru
Hi,

The first one "Hive Metastore Connection Timeout" is the one you should try. If you look closely, "hive.metastore.client.socket.timeout" is just underneath it.

Regarding the JIRA ID, it is an internal JIRA, so you do not have access to it.

Another way is to setup a quick script to drop partition in batches and and then drop the table after number of partitions have reduced to a reasonable level.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Guru

Hi Vibin,

 

I can see that this issue is tracked under HIVE-6980 on the upstream JIRA. And I can confirm that it has been fixed in CDH6.1 and CDH5.16 onwards.

 

If you are using older version, you can try to increase the value for hive.metastore.client.socket.timeout as a workaround as mentioned in the previous post.

 

Hope above is helpful.

 

Cheers

Eric 

12 REPLIES 12

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Expert Contributor
Note: I use Impala Shell v2.9.0-cdh5.12.0 (03c6ddb) built on Thu Jun 29 04:17:31 PDT 2017
Thanks in advance.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Cloudera Employee

Can you try dropping the table using Hive?

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Expert Contributor

Thank you very muth for the answer man

I tried to drop it with the hive, but it give this error message:

hive> drop table cars;
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:hive.metastore.sasl.enabled can't be false in non-testing mode)

I think it's another problem that we have to fix, did you have any idea ?
NB: I use Sentry with hive.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Guru
Hi,

This is a known issue and we have an internal JIRA to track it. Current there is no better way to improve the performance.

You can, however, to increase the timeout limit via "hive.metastore.client.socket.timeout", and set it to long time value (in seconds) to allow the query to finish. It should finish eventually, but might take sometime.

Our engineers are still discussing internally to see the best fix for the issue, however, it is on going and we do not have solution yet at this stage.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Expert Contributor

Hi @EricL,
Thanks for the replay,

Can you give me the JIRA track link of this issue please ?
And about 'hive.metastore.client.socket.timeout' I find 3 config variables with the same name:

1- Hive: Hive Metastore Connection Timeout (5m).
2- Hive: in : Service Monitor Client Config Overrides (value: 60).
3- Impala: Catalog Server Hive Metastore Connection Timeout and Impala Daemon Hive Metastore Connection Timeout (value: 1h).


Can you help me to know which one is the concerned ?


Hope this issue iresolved ASAP.
Thanks again.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Guru
Hi,

The first one "Hive Metastore Connection Timeout" is the one you should try. If you look closely, "hive.metastore.client.socket.timeout" is just underneath it.

Regarding the JIRA ID, it is an internal JIRA, so you do not have access to it.

Another way is to setup a quick script to drop partition in batches and and then drop the table after number of partitions have reduced to a reasonable level.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Expert Contributor

Yes man, that what I do now in *drop* statement, but so far no easy solution for *alter*.
Thanks again @EricL

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

Guru
Glad that I am helpful here.

Re: Can not ALTER or DROP a big partitionned tables - CAUSED BY: MetaException: Timeout when executi

New Contributor

@EricL Just wanted to know if there is any progress on this issue ? I too face the same issue - wanted to drop a table with 50k partitions.