Created on 05-04-2015 07:33 AM - edited 09-16-2022 02:27 AM
Hello,
I'm trying to do an EvalFunc UDF for PIG and I'm getting the folowing error:
java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:275) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:465) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:368) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1477) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1474) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1407)
After searching I found that the mandatory requirements for CDH may not be fulfilled:
http://pig.apache.org/docs/r0.12.0/start.html
En fact, now, I can't see how was this working until now... How is possible to have pig 0.12 with Hadoop 2.5?
Thank you!!
Created on 05-06-2015 01:50 AM - edited 05-06-2015 01:51 AM
Note:
- the job is run with Map Reduce 2 only if I add a new library to the UDF ==> I get this error only when I add an external library to the UDF.
The job run with MR1 (and a lot of warnings) when I added the library with register in the pig script.
Alina
Created on 05-06-2015 01:50 AM - edited 05-06-2015 01:51 AM
Note:
- the job is run with Map Reduce 2 only if I add a new library to the UDF ==> I get this error only when I add an external library to the UDF.
The job run with MR1 (and a lot of warnings) when I added the library with register in the pig script.
Alina