Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hive drop table error

hive drop table error

I am trying to drop a table and below error is thrown. How do I recover from this error ?

drop table <table_name>

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Invalid partition key & values; keys [], values [2450821, ])

complete log

2016-09-28 13:13:47,554 ERROR [main]: exec.DDLTask (DDLTask.java:failed(533)) - org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Invalid partition key & values; keys [], values [2450821, ])

        at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1083)

        at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1014)

        at org.apache.hadoop.hive.ql.exec.DDLTask.dropTable(DDLTask.java:3998)

        at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3854)

        at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:339)

        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)

        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)

        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1745)

        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1491)

        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1289)

        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1156)

        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1146)

        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)

        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)

        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)

        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)

        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)

        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.hadoop.util.RunJar.run(RunJar.java:233)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:148)

Caused by: MetaException(message:Invalid partition key & values; keys [], values [2450821, ])

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_with_environment_context_result$drop_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:40236)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_with_environment_context_result$drop_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:40213)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_with_environment_context_result.read(ThriftHiveMetastore.java:40155)

        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_drop_table_with_environment_context(ThriftHiveMetastore.java:1150)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.drop_table_with_environment_context(ThriftHiveMetastore.java:1134)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.drop_table_with_environment_context(HiveMetaStoreClient.java:2128)

        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.drop_table_with_environment_context(SessionHiveMetaStoreClient.java:118)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:994)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:930)

        at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:159)

        at com.sun.proxy.$Proxy5.dropTable(Unknown Source)

        at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1077)

        ... 23 more
6 REPLIES 6

Re: hive drop table error

Super Guru
@Santhosh B Gowda

Can you please share complete log for this?

Re: hive drop table error

@mqureshi I have updated the question with logs as well

Re: hive drop table error

Super Guru

@Santhosh B Gowda Can you try using mr as your hive execution engine? Just wanted to check if there may be a bug?

Re: hive drop table error

2016-09-28 13:52:57,422 ERROR [main]: exec.DDLTask (DDLTask.java:failed(533)) - org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Invalid partition key & values; keys [], values [2450821, ])

        at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1083)

        at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1014)

        at org.apache.hadoop.hive.ql.exec.DDLTask.dropTable(DDLTask.java:3998)

        at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3854)

        at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:339)

        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)

        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)

        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1745)

        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1491)

        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1289)

        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1156)

        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1146)

        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)

        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)

        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)

        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)

        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)

        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.hadoop.util.RunJar.run(RunJar.java:233)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:148)

Caused by: MetaException(message:Invalid partition key & values; keys [], values [2450821, ])

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_with_environment_context_result$drop_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:40236)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_with_environment_context_result$drop_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:40213)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_with_environment_context_result.read(ThriftHiveMetastore.java:40155)

        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_drop_table_with_environment_context(ThriftHiveMetastore.java:1150)

        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.drop_table_with_environment_context(ThriftHiveMetastore.java:1134)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.drop_table_with_environment_context(HiveMetaStoreClient.java:2128)

        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.drop_table_with_environment_context(SessionHiveMetaStoreClient.java:118)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:994)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:930)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:159)

        at com.sun.proxy.$Proxy5.dropTable(Unknown Source)

        at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1077)

        ... 23 more

Same error

Re: hive drop table error

@Santhosh B Gowda

When a drop is triggered for a partitioned table, it drops the partitions one by one. Seems like one of your partition retrieval has failed: partition key seems to be empty. So, to overcome this I would suggest below..

Try creating a temporary database and move the table 'as is' into the new database.

CREATE DATABASE ifnot exists TemporaryDB;
USE targetDB;
ALTER TABLE TargetTable RENAME TO TemporaryDB.TargetTable;
DROP DATABASE TemporaryDB Cascade;

Re: hive drop table error

@Ayub Pathan even alter table is failing with the same error.