Reply
New Contributor
Posts: 8
Registered: ‎01-03-2014

Running MR examples using YARN framework in Cloudera QuickStart VM

I am trying to run "teragen" program using YARN framework in CDH 4.5 but no luck.
What i have figured out till now:

1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
2. You can either run MRv1 or MRv2 but not both.
3. I disabled MRv1 and configured YARN by increasing the alternatives priority for YARN, deployed the Client Configuration (Using Cloudera Manager).
4. Submitted the teragen program and can see the YARN framework is in action from Web UI's.
5. But it hangs at Map 0% and Reduce %.

Tried multiple option to no avail.

Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?

Attached doc with screen shots.

Reference:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managi...

Regards
Gaurav

Expert Contributor
Posts: 126
Registered: ‎11-01-2013

Re: Running MR examples using YARN framework in Cloudera QuickStart VM

Expert Contributor
Posts: 126
Registered: ‎11-01-2013

Re: Running MR examples using YARN framework in Cloudera QuickStart VM

P.S
Running Teragen or Terasort will not give you good performance results with VM's...You will be using same vm on the same disk your main operating system runs....reading, writing, soring, spilling records on the same disk might not give you good results.
Benchmarking tools are recommended to test the peak performance on a physical cluster...
Em Jay
New Contributor
Posts: 8
Registered: ‎01-03-2014

Re: Running MR examples using YARN framework in Cloudera QuickStart VM

As of now i just need to run a MR Application/Program in YARN for demo purpose. So, testeda sample program. Tried wordcount too but that too didn't move post JOb Submission. These examples run perfectly fine with Mrv1 mode in CDH 4.5.

New Contributor
Posts: 8
Registered: ‎01-03-2014

Re: Running MR examples using YARN framework in Cloudera QuickStart VM

Nothing helped. The basic Requirment is to Code and execute a Wordcount program in MRv2 running over YARN framework.

New Contributor
Posts: 6
Registered: ‎03-18-2014

Re: Running MR examples using YARN framework in Cloudera QuickStart VM