Support Questions

Find answers, ask questions, and share your expertise

When Reducer Operation actually starts..I mean with respect to Copy Phase ?After completion of copy phase or while copy phase is going on???

avatar
Expert Contributor
1 ACCEPTED SOLUTION

avatar
Master Mentor

@suresh bonam One of the best answers http://stackoverflow.com/questions/11672676/when-d...

Reducers start shuffling based on a threshold of percentage of mappers that have finished. You can change the parameter to get reducers to start sooner or later.

Why is starting the reducers early a good thing? Because it spreads out the data transfer from the mappers to the reducers over time, which is a good thing if your network is the bottleneck

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@suresh bonam One of the best answers http://stackoverflow.com/questions/11672676/when-d...

Reducers start shuffling based on a threshold of percentage of mappers that have finished. You can change the parameter to get reducers to start sooner or later.

Why is starting the reducers early a good thing? Because it spreads out the data transfer from the mappers to the reducers over time, which is a good thing if your network is the bottleneck

avatar
Master Mentor

@Suresh Bonam are you still having issues with this? Can you accept best answer or provide your own solution?