Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive (on tez) : Hive Runtime Error while closing operators: Unable to rename output from

Labels:
avatar
author-rank gnanasekaran_g
Contributor

Created ‎08-29-2017 09:29 AM

Hi All,

I have HDP 2.3.2 in production cluster and I have some INSERT-SELECT query to run to populate data from a set of tables to a table and since last week, I get a weird issue - java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://

How should I attack this problem to solve it permanently?
Should I run this query with MR engine rather than Tez? Please throw me some light.

Map 1: 2/2	Map 12: 2/2	Map 14: 2/2	Map 16: 2/2	Map 18: 3/3	Map 19: 2/2	Map 21: 3/3	Map 25: 2/2	Map 26: 2/2	Map 27: 3/3	Map 3: 2/2	M\


ap 5: 3/3	Reducer 10: 1/1	Reducer 11: 0(+1,-3)/111	Reducer 13: 1/1	Reducer 15: 1/1	Reducer 17: 1/1	Reducer 2: 1/1	Reducer 20: 1/1	R\


educer 22: 37/37	Reducer 23: 1/1	Reducer 24: 412/412	Reducer 4: 1/1	Reducer 7: 221/221	Reducer 9: 221/221	


Status: Failed


Vertex failed, vertexName=Reducer 11, vertexId=vertex_1503954544552_1870_1_24, diagnostics=[Task failed, taskId=task_1503954544552_\


1870_1_24_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.\


RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/\


crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000\


000_0 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_79\


42865519677265277-1/_tmp.-ext-10002/000000_0


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)


	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)


	at java.security.AccessController.doPrivileged(Native Method)


	at javax.security.auth.Subject.doAs(Subject.java:422)


	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)


	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)


	at java.util.concurrent.FutureTask.run(FutureTask.java:266)


	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)


	at java.lang.Thread.run(Thread.java:745)


Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_0 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_0


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)


	... 14 more


Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_0 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_0


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)


	... 15 more


], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runti\


me Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_co\


mmentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000000_1 to: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_tmp.-ex\


t-10002/000000_1


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)


	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)


	at java.security.AccessController.doPrivileged(Native Method)


	at javax.security.auth.Subject.doAs(Subject.java:422)


	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)


	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)


	at java.util.concurrent.FutureTask.run(FutureTask.java:266)


	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)


	at java.lang.Thread.run(Thread.java:745)


Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_1 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_1


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)


	... 14 more


Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_1 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_1


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)


	... 15 more


], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runti\


me Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_co\


mmentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000000_2 to: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_tmp.-ex\


t-10002/000000_2


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)


	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)


	at java.security.AccessController.doPrivileged(Native Method)


	at javax.security.auth.Subject.doAs(Subject.java:422)


	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)


	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)


	at java.util.concurrent.FutureTask.run(FutureTask.java:266)


	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)


	at java.lang.Thread.run(Thread.java:745)


Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_2 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_2


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)


	... 14 more


Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_2 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_2


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)


	... 15 more


], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runti\


me Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_co\


mmentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000000_3 to: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_tmp.-ex\


t-10002/000000_3


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)


	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)


	at java.security.AccessController.doPrivileged(Native Method)


	at javax.security.auth.Subject.doAs(Subject.java:422)


	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)


	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)


	at java.util.concurrent.FutureTask.run(FutureTask.java:266)


	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)


	at java.lang.Thread.run(Thread.java:745)


Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_3 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_3


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)


	... 14 more


Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_3 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_3


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)


	... 15 more


]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:110, Vertex vertex_1503954544552_1870_1_24 [Reducer 1\


1] killed/failed due to:OWN_TASK_FAILURE]


DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0


FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer 11, verte\


xId=vertex_1503954544552_1870_1_24, diagnostics=[Task failed, taskId=task_1503954544552_1870_1_24_000000, diagnostics=[TaskAttempt \


0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while \


closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-\


staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000000_0 to: hdfs://STANCDR1TDH/apps/hive/ware\


house/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_\


0


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)


	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)


	at java.security.AccessController.doPrivileged(Native Method)


	at javax.security.auth.Subject.doAs(Subject.java:422)


	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)


	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)


	at java.util.concurrent.FutureTask.run(FutureTask.java:266)


	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)


	at java.lang.Thread.run(Thread.java:745)


Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_0 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_0


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)


	... 14 more


Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_0 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_0


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)


	... 15 more


], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runti\


me Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_co\


mmentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000000_1 to: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_tmp.-ex\


t-10002/000000_1


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)


	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)


	at java.security.AccessController.doPrivileged(Native Method)


	at javax.security.auth.Subject.doAs(Subject.java:422)


	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)


	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)


	at java.util.concurrent.FutureTask.run(FutureTask.java:266)


	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)


	at java.lang.Thread.run(Thread.java:745)


Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_1 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_1


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)


	... 14 more


Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_1 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_1


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)


	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)


	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)


	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)


	... 15 more


], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runti\


me Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_co\


mmentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000000_2 to: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_tmp.-ex\


t-10002/000000_2


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)


	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)


	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)


	at java.security.AccessController.doPrivileged(Native Method)


	at javax.security.auth.Subject.doAs(Subject.java:422)


	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)


	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)


	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)


	at java.util.concurrent.FutureTask.run(FutureTask.java:266)


	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	

at java.lang.Thread.run(Thread.java:745)
	

Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_2 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_2


at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)
	

at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
	

... 14 more
	

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_2 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_2


at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)
	

at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)
	

at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)
	

at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
	

at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
	

at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)
	

... 15 more
	

], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runti\


me Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_co\


mmentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.000000_3 to: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_tmp.-ex\


t-10002/000000_3


at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
	

at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
	

at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
	

at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
	

at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
	

at java.security.AccessController.doPrivileged(Native Method)
	

at javax.security.auth.Subject.doAs(Subject.java:422)
	

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	

at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
	

at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
	

at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	

at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	

at java.lang.Thread.run(Thread.java:745)
	

Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://STANCDR1TDH\


/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tm\


p.-ext-10002/_tmp.000000_3 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-\


08-29_14-34-17_200_7942865519677265277-1/_tmp.-ext-10002/000000_3


at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)
	

at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
	

... 14 more
	

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://STANCDR1TDH/apps/hive/warehouse/c\


rhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_7942865519677265277-1/_task_tmp.-ext-10002/_tmp.0000\


00_3 to: hdfs://STANCDR1TDH/apps/hive/warehouse/crhsctl.db/crhs_sci_break_commentary/.hive-staging_hive_2017-08-29_14-34-17_200_794\


2865519677265277-1/_tmp.-ext-10002/000000_3


at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:237)
	

at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$200(FileSinkOperator.java:143)
	

at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1051)
	

at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
	

at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
	

at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)
	

... 15 more
	

]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:110, Vertex vertex_1503954544552_1870_1_24 [Reducer 1\


1] killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
9,236 Views
0 Kudos
1 REPLY 1

avatar
author-rank SQLShaw author-rank
Guru

Created ‎09-04-2017 08:27 PM

Hi @Gnanasekaran G, is there an OVERRIDE command in your statement? You may be running into this issue - https://issues.apache.org/jira/browse/HIVE-4605

6,229 Views
0 Kudos
Announcements
What's New @ Cloudera
Accelerate Your AI with the Cloudera AI Inference Service wi...
What's New @ Cloudera
Updates to the HDFS clusters on AWS environments to add supp...
What's New @ Cloudera
Enhancements to the create-database command
Community Announcements
September 2024 Community Highlights
What's New @ Cloudera
Cloudera Streams Messaging Operator 1.1
View More Announcements