Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Getting error while running a job

Getting error while running a job

New Contributor
017-03-06 15:20:07,994 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2998: Unhandled internal error. Vertex failed, vertexName=scope-528, vertexId=vertex_1486843207585_727267_1_11, diagnostics=[Vertex vertex_1486843207585_727267_1_11 [scope-528] killed/failed due to:AM_USERCODE_FAILURE, Exception in VertexManager, vertex:vertex_1486843207585_727267_1_11 [scope-528], org.apache.tez.dag.api.TezUncheckedException: org.apache.pig.impl.plan.VisitorException: ERROR 0: java.io.IOException: Estimated parallelism for scope-478 is 0 which is unexpected
at org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigGraceShuffleVertexManager.onVertexStateUpdated(PigGraceShuffleVertexManager.java:162)
at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEventOnVertexStateUpdate.invoke(VertexManager.java:573)
at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:658)
at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:653)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:653)
at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:642)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 0: java.io.IOException: Estimated parallelism for scope-478 is 0 which is unexpected
at org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.ParallelismSetter.visitTezOp(ParallelismSetter.java:192)
at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:249)
at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:55)
at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:87)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46)
at org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigGraceShuffleVertexManager.onVertexStateUpdated(PigGraceShuffleVertexManager.java:159)
... 12 more
Caused by: java.io.IOException: Estimated parallelism for scope-478 is 0 which is unexpected
at org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.TezOperDependencyParallelismEstimator.estimateParallelism(TezOperDependencyParallelismEstimator.java:149)
at org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.ParallelismSetter.visitTezOp(ParallelismSetter.java:129)
... 17 more
2 REPLIES 2

Re: Getting error while running a job

New Contributor
Vertex killed, vertexName=scope-492, vertexId=vertex_1486843207585_727267_1_09, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1486843207585_727267_1_09 [scope-492] killed/failed due to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=scope-479, vertexId=vertex_1486843207585_727267_1_07, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:999, Vertex vertex_1486843207585_727267_1_07 [scope-479] killed/failed due to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=scope-468, vertexId=vertex_1486843207585_727267_1_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:22, Vertex vertex_1486843207585_727267_1_00 [scope-468] killed/failed due to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=scope-478, vertexId=vertex_1486843207585_727267_1_06, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:322, Vertex vertex_1486843207585_727267_1_06 [scope-478] killed/failed due to:OTHER_VERTEX_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:20

Re: Getting error while running a job

@Vijay

The above is just a symptom, not useful to determine the cause. I can only assume that you were running a Hive job. Please look through Hive logs to grep for a recent ERROR. Those logs should be in /var/log/hive or /var/log/hive2, whatever is applicable in your case. Most often it could be lack of system resources and YARN settings that prevent execution when resources are not applicable.

Don't have an account?
Coming from Hortonworks? Activate your account here