Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to see data in riskfactor in step - 3.4.3 Verify Pig Script Successfully Populated Hive Table

Unable to see data in riskfactor in step - 3.4.3 Verify Pig Script Successfully Populated Hive Table

New Contributor

I ran the Pig script as shown in 3.4.1 and it said that it's completed. but I see errors in Logs and don't see any data in riskfactor.

FYI - I am also unable to see Tez from the view.

logs:

WARNING: Use "yarn jar" to launch YARN applications.
16/03/17 19:15:05 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/03/17 19:15:05 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/03/17 19:15:05 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
16/03/17 19:15:05 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
16/03/17 19:15:05 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2016-03-17 19:15:05,352 [main] INFO  org.apache.pig.Main - Apache Pig version 0.15.0.2.4.0.0-169 (rexported) compiled Feb 10 2016, 07:50:04
2016-03-17 19:15:05,353 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1458233186328_0002/container_e20_1458233186328_0002_01_000002/pig_

1458242105348.log
2016-03-17 19:15:06,781 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2016-03-17 19:15:06,955 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2016-03-17 19:15:08,358 [main] INFO  org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-dc74bd83-f1f1-4d38-8a82-c8b538f3106d
2016-03-17 19:15:09,109 [main] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2016-03-17 19:15:09,451 [main] INFO  org.apache.pig.backend.hadoop.ATSService - Created ATS Hook
2016-03-17 19:15:10,788 [main] WARN  org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-03-17 19:15:10,840 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-03-17 19:15:10,873 [main] WARN  org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-03-17 19:15:10,975 [main] INFO  hive.metastore - Connected to metastore.
2016-03-17 19:15:12,838 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve sum using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 5, column 42> Failed to generate logical plan. Nested exception: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve sum using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1776)
	at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1484)
	at org.apache.pig.PigServer.parseAndBuild(PigServer.java:428)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:453)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:439)
	at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
	at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
	at org.apache.pig.Main.run(Main.java:502)
	at org.apache.pig.Main.main(Main.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: 
<file script.pig, line 5, column 42> Failed to generate logical plan. Nested exception: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve sum using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:1572)
	at org.apache.pig.parser.LogicalPlanGenerator.func_eval(LogicalPlanGenerator.java:9403)
	at org.apache.pig.parser.LogicalPlanGenerator.projectable_expr(LogicalPlanGenerator.java:11082)
	at org.apache.pig.parser.LogicalPlanGenerator.var_expr(LogicalPlanGenerator.java:10841)
	at org.apache.pig.parser.LogicalPlanGenerator.expr(LogicalPlanGenerator.java:10190)
	at org.apache.pig.parser.LogicalPlanGenerator.flatten_generated_item(LogicalPlanGenerator.java:7519)
	at org.apache.pig.parser.LogicalPlanGenerator.generate_clause(LogicalPlanGenerator.java:17621)
	at org.apache.pig.parser.LogicalPlanGenerator.foreach_plan(LogicalPlanGenerator.java:16013)
	at org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:15880)
	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1933)
	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
	... 17 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve sum using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:677)
	at org.apache.pig.impl.PigContext.getClassForAlias(PigContext.java:793)
	at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:1569)
	... 30 more
2016-03-17 19:15:12,854 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve sum using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1458233186328_0002/container_e20_1458233186328_0002_01_000002/pig_1458242105348.log
2016-03-17 19:15:12,890 [main] INFO  org.apache.pig.Main - Pig script completed in 7 seconds and 857 milliseconds (7857 ms)

please let me know as what could be the problem.
Thanks
3 REPLIES 3

Re: Unable to see data in riskfactor in step - 3.4.3 Verify Pig Script Successfully Populated Hive Table

Mentor

please post your script. @Umer Malik

Highlighted

Re: Unable to see data in riskfactor in step - 3.4.3 Verify Pig Script Successfully Populated Hive Table

The following line in your output above identifies the problem.

2016-03-1719:15:12,838[main] ERROR org.apache.pig.PigServer- exception during parsing:Error during parsing. Could not resolve sum using imports:[, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

Specifically "Could not resolve sum using imports". The problem is that SUM is a function and as http://pig.apache.org/docs/r0.15.0/basic.html#case-sensitivity indicates, they must be in uppercase.

As for the Tez view not working, please refer to the HCC question at https://community.hortonworks.com/questions/21764/hdp-240-sandbox-integration-of-hive-view-and-tez-v... for more details.

Re: Unable to see data in riskfactor in step - 3.4.3 Verify Pig Script Successfully Populated Hive Table

New Contributor

It worked by itself

Thanks,

Malik

Don't have an account?
Coming from Hortonworks? Activate your account here