Member since
01-18-2016
25
Posts
14
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1733 | 02-26-2016 09:06 PM | |
2206 | 01-19-2016 07:31 PM |
02-26-2016
09:06 PM
1 Kudo
Its not resolved admin team would be looking into it...you can close this issue
... View more
02-26-2016
07:39 PM
1 Kudo
Its an environment error
... View more
02-01-2016
06:16 PM
I tried both in my home driectory and it worked fine and also it worked fine when i had my parm file on s3. But its not working when i try to do it from hdfs.
... View more
01-26-2016
09:00 PM
1 Kudo
pig -x mapreduce /home/hadoop/test.pig -param_file=hdfs://ip-XXX-XX-XX-XXX.ec2.internal:8020/home/hadoop/adh_time I am trying to run the above command and its not working saying undefined parameter i tired multiple ways and nothing works.Any help would be appreciated. Value inside the adh_time is input = 2015-06-16T14:59:55.000Z Error: 16/01/26 20:56:38 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/01/26 20:56:38 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/01/26 20:56:38 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
46 [main] INFO org.apache.pig.Main - Apache Pig version 0.14.0-amzn-0 (r: unknown) compiled Nov 16 2015, 21:51:33
16/01/26 20:56:38 INFO pig.Main: Apache Pig version 0.14.0-amzn-0 (r: unknown) compiled Nov 16 2015, 21:51:33
47 [main] INFO org.apache.pig.Main - Logging error messages to: /mnt/var/log/pig/pig_1453841798872.log
16/01/26 20:56:38 INFO pig.Main: Logging error messages to: /mnt/var/log/pig/pig_1453841798872.log
789 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found
16/01/26 20:56:39 INFO util.Utils: Default bootup file /home/hadoop/.pigbootup not found
810 [main] ERROR org.apache.pig.impl.PigContext - Undefined parameter : input
16/01/26 20:56:39 ERROR impl.PigContext: Undefined parameter : input
828 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. org.apache.pig.tools.parameters.ParameterSubstitutionException: Undefined parameter : input
16/01/26 20:56:39 ERROR pig.Main: ERROR 2997: Encountered IOException. org.apache.pig.tools.parameters.ParameterSubstitutionException: Undefined parameter : input
Details at logfile: /mnt/var/log/pig/pig_1453841798872.log
848 [main] INFO org.apache.pig.Main - Pig script completed in 952 milliseconds (952 ms)
16/01/26 20:56:39 INFO pig.Main: Pig script completed in 952 milliseconds (952 ms)
... View more
- Tags:
- Data Processing
- Pig
Labels:
- Labels:
-
Apache Pig
01-20-2016
09:31 PM
1 Kudo
A = load 's3://xxxx-analytics-etl/etl_datasource/Count_2015-06-20.par' USING parquet.pig.ParquetLoader();
... View more
01-20-2016
09:09 PM
1 Kudo
If i try to run the same script again it works fine
... View more
01-20-2016
09:06 PM
1 Kudo
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. I tried registering all the jar files mentioned but still getting the same error.
... View more
Labels:
- Labels:
-
Apache Pig
01-20-2016
12:31 AM
Thanks Artem the PigStorage(',') worked
... View more
01-19-2016
07:31 PM
1 Kudo
Actually they gave me a corrupted file which was causing the issue...between i have another question can i store the output from parquet as csv ?
... View more
01-18-2016
11:48 PM
A = load 's3://xxs-analytics-etl/event_store/xxxx_2015-06-15' USING parquet.pig.ParquetLoader() as (user: chararray); I tried just reading one column but got the same error
... View more
01-18-2016
11:00 PM
tried it got the same error
... View more
01-18-2016
07:54 PM
I am ok casting the field to string---- Below is my script A = load 's3://XXXX-analytics-etl/event_store/Count_2015-06-15' USING parquet.pig.ParquetLoader() as (class: chararray,localTime: chararray,updated: chararray,tzone: double,rid: chararray,value: double,time: chararray,user: chararray,patchId: long);
... View more
01-18-2016
07:40 PM
i am using parquet-pig-bundle-1.5.0.jar.
... View more
01-18-2016
07:26 PM
Pig Stack Trace
---------------
ERROR 1200: can't convert optional int96 uploadTime
Failed to parse: can't convert optional int96 uploadTime
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:201)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1707)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1680)
at org.apache.pig.PigServer.registerQuery(PigServer.java:623)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1061)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
at org.apache.pig.Main.run(Main.java:560)
at org.apache.pig.Main.main(Main.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: parquet.pig.SchemaConversionException: can't convert optional int96 uploadTime
at parquet.pig.PigSchemaConverter.convertFields(PigSchemaConverter.java:108)
at parquet.pig.PigSchemaConverter.convert(PigSchemaConverter.java:84)
at parquet.pig.TupleReadSupport.getPigSchemaFromMultipleFiles(TupleReadSupport.java:70)
at parquet.pig.ParquetLoader.initSchema(ParquetLoader.java:204)
at parquet.pig.ParquetLoader.setInput(ParquetLoader.java:108)
at parquet.pig.ParquetLoader.getSchema(ParquetLoader.java:188)
at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:901)
at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
... 16 more
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 0: NYI
at parquet.pig.PigSchemaConverter$1.convertINT96(PigSchemaConverter.java:148)
at parquet.pig.PigSchemaConverter$1.convertINT96(PigSchemaConverter.java:120)
at parquet.schema.PrimitiveType$PrimitiveTypeName$7.convert(PrimitiveType.java:219)
at parquet.pig.PigSchemaConverter.getSimpleFieldSchema(PigSchemaConverter.java:119)
at parquet.pig.PigSchemaConverter.getFieldSchema(PigSchemaConverter.java:222)
at parquet.pig.PigSchemaConverter.convertFields(PigSchemaConverter.java:99)
... 30 more
================================================================================
... View more
- Tags:
- Data Processing
- error
Labels:
- Labels:
-
Apache Hadoop
-
Apache Pig