Support Questions
Find answers, ask questions, and share your expertise

In Mapreduce how to sort intermediate output based on values?

How to sort intermediate output based on values In MapReduce?


Expert Contributor
@Dukool SHarma

The MapReduce sort the intermediate data(between mapper and reducer phase) by key by default. If we want the data should be sort based on value, then we need secondary sorting.

For more Information you can reference below links:

Please accept the answer you found most useful.

Sorting is carried out at the Map side. When all the map outputs have been copied, the reduce task moves into the sort phase i.e.maerging phase. which merges the map outputs, maintaining their sort ordering. This is done in rounds. For example, if there were 60 map outputs and the merge factor was 15 (the default, controlled by the property, just like in the map’s merge), there would be four rounds. Each round would merge 15 files into 1, so at the end, there would be 4 intermediate files to be processed. This is done using a key-value pair.