Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

How to Reduce the data volume during shuffling between Mapper and Reducer Node ?

How to Reduce the data volume during shuffling between Mapper and Reducer Node ?

2 REPLIES 2

Improve the performance of data transfer between Mapper and Reducer is by using the Combiner function. Combiner works as a mini reducer which operates on data generated by Mapper and used for the purpose of optimization.

2nd option is we can compress the intermediate output generated by Mapper with the below command in driver class

Explorer

Hello @Harshali Patel, did you see my answer here?

I hope this helps.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.