Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pig stuck at 0% -- problem configuring

Highlighted

Pig stuck at 0% -- problem configuring

New Contributor

Dear Community,

 

Please help. After reading tons of posts and pages, and making memory adjustments and other manipulations, Pig example is stuck at 0%. Please any ideas?

 

Several discussions suggested adding more memory to YARN, did not help. Cluster: 1 server with 16GB RAM and 3 servers with 8GB RAM.

The stock Pig example does not work neither from Hue nor from Grunt.

 

This is the example:

data = LOAD '/user/hue/pig/examples/data/midsummer.txt' as (text:CHARARRAY);
upper_case = FOREACH data GENERATE UPPER(text);
STORE upper_case INTO '$output' ;

 

This is the last line of output:

2015-10-24 02:12:24,095 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete

 

 

Thank you,

Alex

3 REPLIES 3
Highlighted

Re: Pig stuck at 0% -- problem configuring

New Contributor
I had the same problem, you have managed to fix it?
Highlighted

Re: Pig stuck at 0% -- problem configuring

New Contributor
Yes, I did remove Cloudera Hadoop and installed Hortonworks Hadoop. I've
struggled with Cloudera for 2 weeks but Hortonworks was up and running
in one day...

Re: Pig stuck at 0% -- problem configuring

Contributor

Hi Grenoble, Alxrud,

 

Thanks for bringing this to our attention.  Given the details of the reported problem, I attempted to reproduce the same issue spinning up a cluster, and was able to successfully run the Pig script in Hue (as shown below, including cluster configuration steps):

 

Repro details:

1) Setup test cluster with Cloudera Express CM 5.4.1, CDH 5.3.0 (Parcels)

2) Configured the Core Hadoop services in the following fashion (for testing):
    Master (16GB RAM): CM, NN, SNN, Hue, Sqoop, RM, JHS, Hive Gateway
    Worker 1 (8GB RAM): DN, NM, ZK, Hive Gateway, HMS
    Worker 2 (8GB RAM): DN, NM, ZK, Hive Gateway, Oozie
    Worker 3 (8GB RAM): DN, NM, ZK, Hive Gateway, HS2
3) Installed All Hue application examples as Hue admin user
4) Created regular user account in Hue
5) Logged in as regular (non-admin) user in Hue
6) Ran the test query via Hue -> Query Editors -> Pig -> Pasted the following output:
   data = LOAD '/user/hue/pig/examples/data/midsummer.txt' as (text:CHARARRAY);
   upper_case = FOREACH data GENERATE UPPER(text);
   STORE upper_case INTO '$output' ;
 
7) Clicked on Submit
8) Output filename = test3
9) Confirmed workflow output was successful:
 
2016-02-12 09:02:21,226 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 0% complete
2016-02-12 09:02:43,143 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 50% complete
Heart beat
2016-02-12 09:02:46,386 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation  - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2016-02-12 09:02:46,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 100% complete
 
 
More Information Needed: 
To better understand the cause of the reported behavior, kindly provide responses to the following:
 
1) How are all the services distributed on your cluster, and how much memory is allocated for each? 
 
2) Are you able to run a simple Pi job successfully?  
 
For Parcel installs:
$ hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 5 5
 
For package-based installs:
$ hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 5 5
 
3) Can you provide the Resource Manager service log along with the Application Master log of the Pig job that is reportedly hanging?
 
Gathering the above information will help help narrow down components where the culprit resides.  If a simple Pi job does not work, then further attention is needed on the YARN configuration and ensuring that the AM, map (and reduce if applicable) containers are properly launched. 
 
 
Don't have an account?
Coming from Hortonworks? Activate your account here