Pig 0.12 + CDH 5.3 Compatibility

AlinaGHERMAN — Fri, 16 Sep 2022 09:27:50 GMT

Hello,

I'm trying to do an EvalFunc UDF for PIG and I'm getting the folowing error:

java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:275)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:465)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:368)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1477)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1474)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1407)

After searching I found that the mandatory requirements for CDH may not be fulfilled:

http://pig.apache.org/docs/r0.12.0/start.html

http://stackoverflow.com/questions/21873050/pig-found-interface-org-apache-hadoop-mapreduce-jobcontext-but-class-was-expe

En fact, now, I can't see how was this working until now... How is possible to have pig 0.12 with Hadoop 2.5?

Thank you!!

Re: Pig 0.12 + CDH 5.3 Compatibility

AlinaGHERMAN — Wed, 06 May 2015 08:51:31 GMT

Note:

- the job is run with Map Reduce 2 only if I add a new library to the UDF ==> I get this error only when I add an external library to the UDF.

The job run with MR1 (and a lot of warnings) when I added the library with register in the pig script.

Alina

question Re: Pig 0.12 + CDH 5.3 Compatibility in Archives of Support Questions (Read Only)

Pig 0.12 + CDH 5.3 Compatibility

Re: Pig 0.12 + CDH 5.3 Compatibility