Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error running Pig 0.12, CDH 5.3.3 and MR1

Error running Pig 0.12, CDH 5.3.3 and MR1

New Contributor

I am trying to read a rcfile table (2 columns, both string) created in Hive using Pig and piggybank, but I keep running into this error:

 

Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997: Unable to recreate exception from backed error: Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:217)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:151)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:428)
        at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
        at org.apache.pig.PigServer.storeEx(PigServer.java:978)
        at org.apache.pig.PigServer.store(PigServer.java:942)
        at org.apache.pig.PigServer.openIterator(PigServer.java:855)

 

I have registered all jar files in my pig script and tried recompiling pig using the info on this post: http://stackoverflow.com/questions/21873050/pig-found-interface-org-apache-hadoop-mapreduce-jobconte... but nothing seems to work.

 

Our pig was installed from the Cloudera tarballs and we are currently using MR1; I do understand the offending class turned into an interface in hadoop 2.x but I thought using the CDH5 distribution tarballs would keep us out of trouble.  Any help or tip will be greatly appreciated.

Thank you!

4 REPLIES 4

Re: Error running Pig 0.12, CDH 5.3.3 and MR1

Master Guru
How are you invoking Pig, via Grunt shell or via a custom Java code (i.e. PigServer)? If the latter, are you ensuring to pull down CDH versions of the Pig dependencies, and not an Apache Pig vanilla variant (which are built for hadoop1)?

Was any of the tar-balls used built manually, and if so, was -Phadoop2 passed to it to ensure the build runs for the newer API?

Re: Error running Pig 0.12, CDH 5.3.3 and MR1

New Contributor

 

Hi Harsh, thanks for the reply.

 

I am invoking via command line "pig script_name.pig", but I get the same error if I use grunt shell.

 

The tar balls were not built manually; they were installed as they were downloaded from cloudera. We are not using an Apache Pig version.

 

 

Re: Error running Pig 0.12, CDH 5.3.3 and MR1

Master Guru
Can you also share a sample script that reproduces the behaviour? It
appears like one of the supplied jars may be causing this, cause C5 expects
an interface but you're getting an inverted exception.

Re: Error running Pig 0.12, CDH 5.3.3 and MR1

New Contributor

Sure. My script is pretty simple:

 

register /cfs/software/CDH5.3.3/hadoop-2.5.0-mr1-cdh5.3.3/hadoop-core-2.5.0-mr1-cdh5.3.3.jar
register /cfs/software/CDH5.3.3/hive-0.13.1-cdh5.3.3/lib/hive-serde-0.13.1-cdh5.3.3.jar
register /cfs/software/CDH5.3.3/hive-0.13.1-cdh5.3.3/lib/hive-common-0.13.1-cdh5.3.3.jar
register /cfs/software/CDH5.3.3/hive-0.13.1-cdh5.3.3/lib/hive-shims-common-0.13.1-cdh5.3.3.jar
register /cfs/software/CDH5.3.3/hive-0.13.1-cdh5.3.3/lib/hive-shims-0.23-0.13.1-cdh5.3.3.jar
register /cfs/software/CDH5.3.3/hive-0.13.1-cdh5.3.3/lib/hive-shims-common-secure-0.13.1-cdh5.3.3.jar
register /cfs/software/CDH5.3.3/hive-0.13.1-cdh5.3.3/lib/hive-exec-0.13.1-cdh5.3.3.jar

SET debug 'on';
SET job.name 'test.pig';


A = LOAD '/temp/test_rcfile' USING org.apache.pig.piggybank.storage.HiveColumnarLoader('visitor_id string,file_date string');

DUMP A;