Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pig job failing on TEZ

Highlighted

Pig job failing on TEZ

New Contributor

---------------

ERROR 2017: Internal error creating job configuration.

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias result

at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1694)

at org.apache.pig.PigServer.registerQuery(PigServer.java:623)

at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1063)

at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)

at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)

at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)

at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)

at org.apache.pig.Main.run(Main.java:558)

at org.apache.pig.Main.main(Main.java:170)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: org.apache.pig.backend.hadoop.executionengine.JobCreationException: ERROR 2017: Internal error creating job configuration.

at org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler.getJob(TezJobCompiler.java:113)

at org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler.compile(TezJobCompiler.java:73)

at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.launchPig(TezLauncher.java:155)

at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:282)

at org.apache.pig.PigServer.launchPlan(PigServer.java:1390)

at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375)

at org.apache.pig.PigServer.execute(PigServer.java:1364)

at org.apache.pig.PigServer.access$500(PigServer.java:113)

at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1689)

... 14 more

Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 0: Cannot create vertex for Tez(6331,StringSize,SUBSTRING,ToDate2ARGS,ToUnixTime,HCatLoader,TRIM) - scope-1285

at org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder.visitTezOp(TezDagBuilder.java:220)

at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:232)

at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:49)

at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:70)

at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46)

at org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler.buildDAG(TezJobCompiler.java:63)

at org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler.getJob(TezJobCompiler.java:108)

... 22 more

Caused by: java.lang.RuntimeException: Serializing to a ByteString threw an IOException (should never happen).

at com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:54)

at org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder.newVertex(TezDagBuilder.java:642)

at org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder.visitTezOp(TezDagBuilder.java:213)

... 28 more

Caused by: com.google.protobuf.CodedOutputStream$OutOfSpaceException: CodedOutputStream was writing to a flat byte array and ran out of space.

at com.google.protobuf.CodedOutputStream.refreshBuffer(CodedOutputStream.java:828)

at com.google.protobuf.CodedOutputStream.writeRawBytes(CodedOutputStream.java:959)

at com.google.protobuf.CodedOutputStream.writeRawBytes(CodedOutputStream.java:905)

at com.google.protobuf.CodedOutputStream.writeBytesNoTag(CodedOutputStream.java:386)

at com.google.protobuf.CodedOutputStream.writeBytes(CodedOutputStream.java:229)

at org.apache.tez.mapreduce.protos.MRRuntimeProtos$MRSplitProto.writeTo(MRRuntimeProtos.java:1173)

at com.google.protobuf.CodedOutputStream.writeMessageNoTag(CodedOutputStream.java:380)

at com.google.protobuf.CodedOutputStream.writeMessage(CodedOutputStream.java:222)

at org.apache.tez.mapreduce.protos.MRRuntimeProtos$MRSplitsProto.writeTo(MRRuntimeProtos.java:282)

at com.google.protobuf.CodedOutputStream.writeMessageNoTag(CodedOutputStream.java:380)

at com.google.protobuf.CodedOutputStream.writeMessage(CodedOutputStream.java:222)

at org.apache.tez.mapreduce.protos.MRRuntimeProtos$MRInputUserPayloadProto.writeTo(MRRuntimeProtos.java:1797)

at com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:51)

... 30 more

================================================================================

3 REPLIES 3
Highlighted

Re: Pig job failing on TEZ

Mentor

Can you provide details how you run your script, sample input, etc.

Re: Pig job failing on TEZ

Also which version of HDP, and whether the script succeeds using MR.

Highlighted

Re: Pig job failing on TEZ

@raju puli

Seems like this is where the problem is:

Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 0: Cannot create vertex for Tez(6331,StringSize,SUBSTRING,ToDate2ARGS,ToUnixTime,HCatLoader,TRIM) - scope-1285

Can you try to run the commands individually in CLI before running it as a job?

Don't have an account?
Coming from Hortonworks? Activate your account here