Reply
New Contributor
Posts: 1
Registered: ‎12-03-2013

Wordcount tutorial: map 0% reduce 0% non moving on!

Hi to all

 

We are trying to apply the first tutorial (wordcount) usign the virtual machine (VMWare versione), but the map/reduce phase not move from 0%!

 

Any suggstion?

 

[cloudera@hadoop ~]$ hadoop jar wordcount.jar org.myorg.WordCount /user/cloudera/wordcount/input /user/cloudera/wordcount/output
13/12/03 16:11:33 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/12/03 16:11:33 INFO mapred.FileInputFormat: Total input paths to process : 2
13/12/03 16:11:34 INFO mapred.JobClient: Running job: job_201312031428_0001
13/12/03 16:11:35 INFO mapred.JobClient:  map 0% reduce 0%

 

Thanks

 

Carlo

Expert Contributor
Posts: 126
Registered: ‎11-01-2013

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

This happens usually when you use VM's because, VM is a machine running on another machine. Hard disk is shared between your real machine and virtual machine. You have data on the same disk and you are trying to perform mapreduce on same disk while real machine might be using disk for some other purposes. Due to this kind of read write transactions on same disk at same time impacts your performance and cause hung. You will not see this interruption with physical machines.
Solution would be restarting job or try to reduce other operations on the system while running the job. Thanks

Em Jay
New Contributor
Posts: 8
Registered: ‎01-03-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

I have the same problem in QuickStart VM. It doesn't work at not. No restart or reconfigure helps. Tried to change the Memory COnfig too but to no avail.

I am trying to run "teragen" program using YARN framework in CDH 4.5 but no luck.
What i have figured out till now:

1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
2. You can either run MRv1 or MRv2 but not both.
3. I disabled MRv1 and configured YARN by increasing the alternatives priority for YARN, deployed the Client Configuration (Using Cloudera Manager).
4. Submitted the teragen program and can see the YARN framework is in action from Web UI's.
5. But it hangs at Map 0% and Reduce %.

Tried multiple option to no avail.

Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?

Attached doc with screen shots.

Reference:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managi...

Regards
Gaurav

New Contributor
Posts: 8
Registered: ‎01-03-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

Nothing helped. The basic Requirment is to Code and execute a Wordcount program in MRv2 running over YARN framework.

Highlighted
New Contributor
Posts: 6
Registered: ‎03-18-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

New Contributor
Posts: 2
Registered: ‎04-23-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

Normally these problems will occur when you are working first time.  Try to rerun the code.  I mean you need to logoff, login and not just rerun at prompt.  Compile your java code and run.

New Contributor
Posts: 6
Registered: ‎05-16-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

Having the same probem, none of these suggestions helped. Even the pi demo hangs. The following output is created in the output directory:

[cloudera@unknown080027e2690f ~]$ dfs -ls output60/_logs/history
Found 2 items
-rw-r--r--   3 cloudera cloudera          0 2014-05-16 12:36 output60/_logs/history/job_201405161234_0001_1400268990342_cloudera_word+count
-rw-r--r--   3 cloudera cloudera      76764 2014-05-16 12:36 output60/_logs/history/job_201405161234_0001_conf.xml
[cloudera@unknown080027e2690f ~]$

Anyone have any additional suggestions?

 

TIA

New Contributor
Posts: 6
Registered: ‎05-16-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

Installed Quickstart VM 5.0 (previous was 4.6) and It doesn't suffer form this problem.

New Contributor
Posts: 6
Registered: ‎05-16-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

Resolved this in 4.6 quickstart VM by restarting mapreduce job from gui, although I didn't see any indication of distress. Woo Hoo! demos run!

New Contributor
Posts: 7
Registered: ‎05-03-2014

Re: Wordcount tutorial: map 0% reduce 0% non moving on!

Are you running through terminal or through eclips ?

As If you ran once through eclipse your namenode will go in a safe mode. And it will block the executiion.

 

Thanks,

-Raj