Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 07-14-2017 08:31 PM
trying to perform a simple test using HIVE + PIG using the following PIG script.
--Load records from populated table int = LOAD 'table1' USING org.apache.hive.hcatalog.pig.HCatLoader(); --store the records to a new table store int into 'table2' using org.apache.hive.hcatalog.pig.HCatStorer(); -useHCatalog added
Any idea on the error? No records get process into the identical and empty table 2.
WARNING: Use "yarn jar" to launch YARN applications.
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
17/07/14 20:08:09 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2017-07-14 20:08:09,110 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.5.0.0-1245 (rexported) compiled Aug 26 2016, 02:07:35
2017-07-14 20:08:09,110 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1499717461125_0015/container_1499717461125_0015_01_000002/pig_1500062889108.log
2017-07-14 20:08:10,228 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2017-07-14 20:08:10,418 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2017-07-14 20:08:11,609 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-edf7096e-5d22-4b5d-a99a-8a1b879aee01
2017-07-14 20:08:12,279 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2017-07-14 20:08:12,493 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2017-07-14 20:08:13,025 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. <file script.pig, line 2, column 0> mismatched input 'int' expecting EOF
Failed to parse: <file script.pig, line 2, column 0> mismatched input 'int' expecting EOF
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:244)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:182)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:503)
at org.apache.pig.Main.main(Main.java:178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
2017-07-14 20:08:13,030 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file script.pig, line 2, column 0> mismatched input 'int' expecting EOF
Details at logfile: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1499717461125_0015/container_1499717461125_0015_01_000002/pig_1500062889108.log
2017-07-14 20:08:13,087 [main] INFO org.apache.pig.Main - Pig script completed in 4 seconds and 218 milliseconds (4218 ms)
Created 07-14-2017 08:54 PM
@John Bowler you cannot use int as the pig relationship name since it is a keyword.
Use anything else like int1 or myint if you like.
That is exactly what your error is about. Change the relationship name and you should be good.
Thanks
Created 07-14-2017 08:54 PM
@John Bowler you cannot use int as the pig relationship name since it is a keyword.
Use anything else like int1 or myint if you like.
That is exactly what your error is about. Change the relationship name and you should be good.
Thanks
Created 07-17-2017 04:49 PM
Big thanks for this!! I really appreciate the help!!
Created 07-17-2017 04:58 PM
No problem @John Bowler. We are here to help. Happy hadooping!!