- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Map reduce flow clarification
- Labels:
-
Apache Hadoop
Created ‎12-30-2016 01:18 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
what is the order of execution for mapreduce Job? Is it correct and please correct me if i am wrong?
Mapper partition each mapper output sorting with in each partition based on key grouping shuffle and merge:each reducer will take one partition from all map tasks and merge together combiner reducer
Created on ‎12-30-2016 09:14 PM - edited ‎08-19-2019 04:57 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created on ‎12-30-2016 09:14 PM - edited ‎08-19-2019 04:57 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created ‎12-31-2016 05:49 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a)could you please provide source for this link and it is really useful
b)what about these two in the diagram and where it will come?
- grouping
- shuffle and merge:each reducer will take one partition from all map tasks and merge together
Created ‎12-31-2016 06:51 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a> This slide is from Hortonworks Training course. The course/slides are available to paid customers only.
b>
i> there is nothing like grouping
ii> Shuffle happens when data move from Map to reduce (please see the diagram) and Merge happens during sort phase at Reducer side.
