question Re: Map reduce flow clarification in Archives of Support Questions (Read Only)

Map reduce flow clarification

vamsi123 — Fri, 30 Dec 2016 21:18:41 GMT

what is the order of execution for mapreduce Job? Is it correct and please correct me if i am wrong?

Mapper
partition each mapper output
sorting with in each partition based on key
grouping
shuffle and merge:each reducer will take one partition from all map tasks and merge together
combiner
reducer

Re: Map reduce flow clarification

manish1 — Mon, 19 Aug 2019 11:57:52 GMT

@vamsi valiveti

This is the right order:

Re: Map reduce flow clarification

vamsi123 — Sat, 31 Dec 2016 13:49:32 GMT

a)could you please provide source for this link and it is really useful

b)what about these two in the diagram and where it will come?

grouping
shuffle and merge:each reducer will take one partition from all map tasks and merge together

Re: Map reduce flow clarification

manish1 — Sat, 31 Dec 2016 14:51:41 GMT

@vamsi valiveti

a> This slide is from Hortonworks Training course. The course/slides are available to paid customers only.

i> there is nothing like grouping

ii> Shuffle happens when data move from Map to reduce (please see the diagram) and Merge happens during sort phase at Reducer side.