Created 08-15-2020 07:57 PM
Getting below error query completes almost to 99 % at the end its getting failed during shuffle phase.Thanks in advance please let me know the reason.
ERROR : Vertex failed, vertexName=Reducer 3, vertexId=vertex_1594037379069_89091_1_02, diagnostics=[Task failed, taskId=task_1594037379069_89091_1_02_000052, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$ShuffleError: error in shuffle in Fetcher_O {Reducer_2} #12
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.callInternal(Shuffle.java:305)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.callInternal(Shuffle.java:287)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Reducer_2: Shuffle failed with too many fetch failures and insufficient progress!failureCounts=5, pendingInputs=5, fetcherHealthy=false, reducerProgressedEnough=true, reducerStalled=true
Created 08-15-2020 09:36 PM
Created 08-15-2020 10:02 PM
Created 08-17-2020 07:30 AM
@Sham Can you collect yarn logs -applicationId application_1594037379069_89091 > app89091.txt and attach it here.