Support Questions

Find answers, ask questions, and share your expertise

can not truncate table

avatar
Contributor

I try to truncate table on Hive

but error Occur

should i do?

hive> truncate table ds_0co_om_cca_1_d_enr_temp; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing hive>

6 REPLIES 6

avatar
Rising Star

@Funamizu Koshi Could you please post the full stack of error message from hivemetastore.log, default location is /var/log/hive

avatar
Rising Star

here is the complete log

18/04/26 23:33:06 [main]: ERROR exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: Exception while processing at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4718) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:516) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1756) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1497) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1294) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1151) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:217) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:169) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:380) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:740) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:685) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1139) at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4716) ... 20 more Caused by:MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41936) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41922) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result.read(ThriftHiveMetastore.java:41872) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_truncate_table(ThriftHiveMetastore.java:1198) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.truncate_table(ThriftHiveMetastore.java:1183) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.truncateTable(HiveMetaStoreClient.java:1065) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.truncateTable(SessionHiveMetaStoreClient.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:178) at com.sun.proxy.$Proxy5.truncateTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1137) ... 21 more 18/04/26 23:33:06 [main]: INFO hooks.ATSHook: Created ATS Hook 18/04/26 23:33:06 [main]: INFO log.PerfLogger: <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver> 18/04/26 23:33:06 [main]: INFO log.PerfLogger: </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1524803586604 end=1524803586604 duration=0 from=org.apache.hadoop.hive.ql.Driver> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing

avatar
Master Guru
@Funamizu Koshi

Can you please start your Hive shell in DEBUG mode and check if you get any relevant error in DEBUG logs?

To start Hive shell in DEBUG mode:

hive --hiveconf hive.root.logger=DEBUG,console

avatar
Rising Star

here is the complete log

18/04/26 23:33:06 [main]: ERROR exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: Exception while processing at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4718) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:516) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1756) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1497) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1294) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1151) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:217) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:169) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:380) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:740) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:685) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1139) at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4716) ... 20 more Caused by:MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41936) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41922) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result.read(ThriftHiveMetastore.java:41872) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_truncate_table(ThriftHiveMetastore.java:1198) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.truncate_table(ThriftHiveMetastore.java:1183) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.truncateTable(HiveMetaStoreClient.java:1065) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.truncateTable(SessionHiveMetaStoreClient.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:178) at com.sun.proxy.$Proxy5.truncateTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1137) ... 21 more 18/04/26 23:33:06 [main]: INFO hooks.ATSHook: Created ATS Hook 18/04/26 23:33:06 [main]: INFO log.PerfLogger: <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver> 18/04/26 23:33:06 [main]: INFO log.PerfLogger: </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1524803586604 end=1524803586604 duration=0 from=org.apache.hadoop.hive.ql.Driver> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing

avatar
Rising Star

what is the solutions for this ..?

avatar
Explorer

Run msck command for the table you want to truncate in hive shell.

hive> use <database-name>;

hive> msck repair table <table-name>;

If it will show any error then rectify it as we got one of our partition was missing.

 

So we create that partition directory on hdfs location and re-run msck repair command. Now it would not show any issue.

 

Now running truncate command will run successfully. 

hive> truncate table <table-name>;

 

[NOTE: Please update database and table name as per the requirement]