Created 07-14-2017 08:31 PM
trying to perform a simple test using HIVE + PIG using the following PIG script.
--Load records from populated table int = LOAD 'table1' USING org.apache.hive.hcatalog.pig.HCatLoader(); --store the records to a new table store int into 'table2' using org.apache.hive.hcatalog.pig.HCatStorer(); -useHCatalog added
Any idea on the error? No records get process into the identical and empty table 2.
WARNING: Use "yarn jar" to launch YARN applications.
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2017-07-14 20:08:09,110 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.5.0.0-1245 (rexported) compiled Aug 26 2016, 02:07:35
2017-07-14 20:08:09,110 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1499717461125_0015/container_1499717461125_0015_01_000002/pig_1500062889108.log
2017-07-14 20:08:10,228 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2017-07-14 20:08:10,418 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2017-07-14 20:08:11,609 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-edf7096e-5d22-4b5d-a99a-8a1b879aee01
2017-07-14 20:08:12,279 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2017-07-14 20:08:12,493 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2017-07-14 20:08:13,025 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. <file script.pig, line 2, column 0> mismatched input 'int' expecting EOF
Failed to parse: <file script.pig, line 2, column 0> mismatched input 'int' expecting EOF
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:244)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:182)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:503)
at org.apache.pig.Main.main(Main.java:178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
2017-07-14 20:08:13,030 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file script.pig, line 2, column 0> mismatched input 'int' expecting EOF
Details at logfile: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1499717461125_0015/container_1499717461125_0015_01_000002/pig_1500062889108.log
2017-07-14 20:08:13,087 [main] INFO org.apache.pig.Main - Pig script completed in 4 seconds and 218 milliseconds (4218 ms)
Created 07-14-2017 08:54 PM
@John Bowler you cannot use int as the pig relationship name since it is a keyword.
Use anything else like int1 or myint if you like.
That is exactly what your error is about. Change the relationship name and you should be good.
Thanks
Created 07-14-2017 08:54 PM
@John Bowler you cannot use int as the pig relationship name since it is a keyword.
Use anything else like int1 or myint if you like.
That is exactly what your error is about. Change the relationship name and you should be good.
Thanks
Created 07-17-2017 04:49 PM
Big thanks for this!! I really appreciate the help!!
Created 07-17-2017 04:58 PM
No problem @John Bowler. We are here to help. Happy hadooping!!