Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Pig 0.12 + CDH 5.3 Compatibility

avatar
Champion Alumni

Hello,

 

I'm trying to do an EvalFunc  UDF for PIG and I'm getting the folowing error:

 

java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:275)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:465)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:368)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1477)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1474)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1407)

 

After searching I found that the mandatory requirements for CDH may not be fulfilled:

http://pig.apache.org/docs/r0.12.0/start.html 

http://stackoverflow.com/questions/21873050/pig-found-interface-org-apache-hadoop-mapreduce-jobconte...

 

 

En fact, now,  I can't see how was this working until now... How is possible to have pig 0.12 with Hadoop 2.5?

 

 

Thank you!!

 

 

 

GHERMAN Alina
1 ACCEPTED SOLUTION

avatar
Champion Alumni

 

Note:

- the job is run with Map Reduce 2 only if I add a new library to the UDF  ==> I get this error only when I add an external library to the UDF. 

 

The job run with MR1 (and a lot of warnings) when I added the library with register in the pig script. 

 

Alina

 

 

GHERMAN Alina

View solution in original post

1 REPLY 1

avatar
Champion Alumni

 

Note:

- the job is run with Map Reduce 2 only if I add a new library to the UDF  ==> I get this error only when I add an external library to the UDF. 

 

The job run with MR1 (and a lot of warnings) when I added the library with register in the pig script. 

 

Alina

 

 

GHERMAN Alina