Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

When Reducer Operation actually starts..I mean with respect to Copy Phase ?After completion of copy phase or while copy phase is going on???

avatar
Expert Contributor
1 ACCEPTED SOLUTION

avatar
Master Mentor

@suresh bonam One of the best answers http://stackoverflow.com/questions/11672676/when-d...

Reducers start shuffling based on a threshold of percentage of mappers that have finished. You can change the parameter to get reducers to start sooner or later.

Why is starting the reducers early a good thing? Because it spreads out the data transfer from the mappers to the reducers over time, which is a good thing if your network is the bottleneck

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@suresh bonam One of the best answers http://stackoverflow.com/questions/11672676/when-d...

Reducers start shuffling based on a threshold of percentage of mappers that have finished. You can change the parameter to get reducers to start sooner or later.

Why is starting the reducers early a good thing? Because it spreads out the data transfer from the mappers to the reducers over time, which is a good thing if your network is the bottleneck

avatar
Master Mentor

@Suresh Bonam are you still having issues with this? Can you accept best answer or provide your own solution?