Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Wordcount tutorial: map 0% reduce 0% non moving on!

Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Hi to all

 

We are trying to apply the first tutorial (wordcount) usign the virtual machine (VMWare versione), but the map/reduce phase not move from 0%!

 

Any suggstion?

 

[cloudera@hadoop ~]$ hadoop jar wordcount.jar org.myorg.WordCount /user/cloudera/wordcount/input /user/cloudera/wordcount/output
13/12/03 16:11:33 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/12/03 16:11:33 INFO mapred.FileInputFormat: Total input paths to process : 2
13/12/03 16:11:34 INFO mapred.JobClient: Running job: job_201312031428_0001
13/12/03 16:11:35 INFO mapred.JobClient:  map 0% reduce 0%

 

Thanks

 

Carlo

9 REPLIES 9

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

Expert Contributor
This happens usually when you use VM's because, VM is a machine running on another machine. Hard disk is shared between your real machine and virtual machine. You have data on the same disk and you are trying to perform mapreduce on same disk while real machine might be using disk for some other purposes. Due to this kind of read write transactions on same disk at same time impacts your performance and cause hung. You will not see this interruption with physical machines.
Solution would be restarting job or try to reduce other operations on the system while running the job. Thanks

Em Jay

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

I have the same problem in QuickStart VM. It doesn't work at not. No restart or reconfigure helps. Tried to change the Memory COnfig too but to no avail.

I am trying to run "teragen" program using YARN framework in CDH 4.5 but no luck.
What i have figured out till now:

1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
2. You can either run MRv1 or MRv2 but not both.
3. I disabled MRv1 and configured YARN by increasing the alternatives priority for YARN, deployed the Client Configuration (Using Cloudera Manager).
4. Submitted the teragen program and can see the YARN framework is in action from Web UI's.
5. But it hangs at Map 0% and Reduce %.

Tried multiple option to no avail.

Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?

Attached doc with screen shots.

Reference:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managi...

Regards
Gaurav

Highlighted

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Nothing helped. The basic Requirment is to Code and execute a Wordcount program in MRv2 running over YARN framework.

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Normally these problems will occur when you are working first time.  Try to rerun the code.  I mean you need to logoff, login and not just rerun at prompt.  Compile your java code and run.

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Having the same probem, none of these suggestions helped. Even the pi demo hangs. The following output is created in the output directory:

[cloudera@unknown080027e2690f ~]$ dfs -ls output60/_logs/history
Found 2 items
-rw-r--r--   3 cloudera cloudera          0 2014-05-16 12:36 output60/_logs/history/job_201405161234_0001_1400268990342_cloudera_word+count
-rw-r--r--   3 cloudera cloudera      76764 2014-05-16 12:36 output60/_logs/history/job_201405161234_0001_conf.xml
[cloudera@unknown080027e2690f ~]$

Anyone have any additional suggestions?

 

TIA

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Installed Quickstart VM 5.0 (previous was 4.6) and It doesn't suffer form this problem.

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Resolved this in 4.6 quickstart VM by restarting mapreduce job from gui, although I didn't see any indication of distress. Woo Hoo! demos run!

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor

Are you running through terminal or through eclips ?

As If you ran once through eclipse your namenode will go in a safe mode. And it will block the executiion.

 

Thanks,

-Raj