Created on 04-27-2017 06:43 AM - edited 09-16-2022 04:31 AM
I try to truncate table on Hive
but error Occur
should i do?
hive> truncate table ds_0co_om_cca_1_d_enr_temp; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing hive>
Created 04-28-2017 05:39 PM
@Funamizu Koshi Could you please post the full stack of error message from hivemetastore.log, default location is /var/log/hive
Created 04-27-2018 09:42 AM
here is the complete log
18/04/26 23:33:06 [main]: ERROR exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: Exception while processing at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4718) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:516) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1756) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1497) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1294) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1151) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:217) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:169) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:380) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:740) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:685) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1139) at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4716) ... 20 more Caused by:MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41936) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41922) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result.read(ThriftHiveMetastore.java:41872) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_truncate_table(ThriftHiveMetastore.java:1198) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.truncate_table(ThriftHiveMetastore.java:1183) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.truncateTable(HiveMetaStoreClient.java:1065) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.truncateTable(SessionHiveMetaStoreClient.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:178) at com.sun.proxy.$Proxy5.truncateTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1137) ... 21 more 18/04/26 23:33:06 [main]: INFO hooks.ATSHook: Created ATS Hook 18/04/26 23:33:06 [main]: INFO log.PerfLogger: <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver> 18/04/26 23:33:06 [main]: INFO log.PerfLogger: </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1524803586604 end=1524803586604 duration=0 from=org.apache.hadoop.hive.ql.Driver> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing
Created 04-29-2017 02:56 AM
Can you please start your Hive shell in DEBUG mode and check if you get any relevant error in DEBUG logs?
To start Hive shell in DEBUG mode:
hive --hiveconf hive.root.logger=DEBUG,console
Created 04-27-2018 09:47 AM
here is the complete log
18/04/26 23:33:06 [main]: ERROR exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: Exception while processing at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4718) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:516) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1756) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1497) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1294) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1151) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:217) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:169) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:380) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:740) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:685) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1139) at org.apache.hadoop.hive.ql.exec.DDLTask.truncateTable(DDLTask.java:4716) ... 20 more Caused by:MetaException(message:Got exception: java.io.IOException Failed to move to trash: hdfs:<> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41936) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result$truncate_table_resultStandardScheme.read(ThriftHiveMetastore.java:41922) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$truncate_table_result.read(ThriftHiveMetastore.java:41872) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_truncate_table(ThriftHiveMetastore.java:1198) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.truncate_table(ThriftHiveMetastore.java:1183) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.truncateTable(HiveMetaStoreClient.java:1065) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.truncateTable(SessionHiveMetaStoreClient.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:178) at com.sun.proxy.$Proxy5.truncateTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.truncateTable(Hive.java:1137) ... 21 more 18/04/26 23:33:06 [main]: INFO hooks.ATSHook: Created ATS Hook 18/04/26 23:33:06 [main]: INFO log.PerfLogger: <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver> 18/04/26 23:33:06 [main]: INFO log.PerfLogger: </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1524803586604 end=1524803586604 duration=0 from=org.apache.hadoop.hive.ql.Driver> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing
Created 04-27-2018 09:47 AM
what is the solutions for this ..?
Created 07-23-2020 07:28 AM
Run msck command for the table you want to truncate in hive shell.
hive> use <database-name>;
hive> msck repair table <table-name>;
If it will show any error then rectify it as we got one of our partition was missing.
So we create that partition directory on hdfs location and re-run msck repair command. Now it would not show any issue.
Now running truncate command will run successfully.
hive> truncate table <table-name>;
[NOTE: Please update database and table name as per the requirement]